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Abstract 

We introduce a domain-independent framework for heterogeneous natural deduction that combines diagrammatic and sen- 
tential reasoning. The framework is presented in the form of a family of denotational proof languages (DPLs). Diagrams 
are represented as possibly partial descriptions of finite system states. This allows us to deal with incomplete information, 
which we formalize by admitting sets as attribute values. We introduce a notion of attribute interpretations that enables 
us to interpret first-order signatures into such system states, and develop a formal semantic framework based on Kleene's 
strong three-valued logic. We extend the assumption-base semantics of DPLs to accodomodate diagrammatic reasoning by 
introducing general inference mechanisms for the valid extraction of information from diagrams and for the incorporation 
of sentential information into diagrams. A rigorous big-step operational semantics is given, on the basis of which we prove 
that our framework is sound. In addition, we specify detailed algorithms for implementing proof checkers for the resulting 
languages, and discuss associated efficiency issues. 

1.1 Introduction 

Diagrams have been recognized as valuable representational and reasoning tools at least since the days of 
Euclid. Their utility is often thought to stem from the fact that diagrams have structural correspondences with 
the objects or situations they represent — they are analogical representations in the celebrated terminology 
of Sloman (Sloman 1971), or homomorphic representations in the terminology of Barwise and Etchemendy 
(Barwise and Etchemendy 1995a). In more plain terms, a diagram resembles what the diagram depicts, in 
contrast to sentential — or "Fregean" (Sloman 1971) — descriptions. This was noticed at least as far back as the 
19th century, when Charles Peirce observed that a diagram is "naturally analogous to the thing represented" 
(Peirce 1960). 

Consider, for instance, the task of descibing some human face. We could perhaps describe the face with 
a collection of English sentences, or with a set of sentences in some formal language. But such a description 
is likely to be excessively long and complicated, and hence not particularly illuminating, yj A drawing or a 
picture of the face, on the other hand, will be much more perspicuous, as well as significantly more compact 
than any sentential representation. Of course, some diagrams are better than others. A talented artist will 
produce a drawing that is a much more accurate depiction than the scrawlings of a 5-year-old. A digital picture 
will be even more accurate^] So, as Hammer observes (Hammer 1995), being an analogical or homomorphic 
representation is not a distinguishing feature of diagrams in general, but rather a distinguishing feature of good 
diagrams. 

This ability of (good) diagrams is in turn often thought to derive from the fact that diagrams are two- 
dimensional objects, and therefore spatial relationships in the diagram can directly reflect analogous relation- 
ships in the underlying domain, an observation made a while back by Russell (Russell 1923). A classic example 
are maps. We can represent the streets of a city graphically, with a map, or sententially, e.g., by a collection 
of assertions expressing the various intersections and so forth. The graphical representation is without doubt a 
more intuitive and effective description because its spatial structure is similar to the actual layout of the city; 
this analogical correspondence is lost in the sentential representation. As another example, consider a map of 
a lake and try to imagine a sentential description of it. Stenning and Lemon (Stenning and Lemon 2001) trace 
this discrepancy to the fact that sentential languages derive from acoustic signals, which are one-dimensional 
and must therefore rely on a complex syntax for representation, something that is not necessary in the case of 
diagrams. 



'Fractals (Manbelbrot 1982) might be able to yield compact representations for some complex shapes such as coastlines, etc., but the 
equations generating the fractals would be no more homomorphic to the corresponding shapes than other sentential descriptions. 

2 In the limiting case, of course, the ultimate representation of an object is the object itself; in that case we have a perfect isomorphism 
between the representation and the object represented. 



Nevertheless, it is important to keep in mind that two-dimensionality by itself is neither a necessary nor a 
sufficient condition for being a diagram. For instance, as Hammer (Hammer 1995) points out, a representation 
of a picture by a two-dimensional array of numbers encoded under some encryption scheme does not classify as 
a diagram; there is no structural similarity between the representation and that which is being represented. And 
by making sufficiently clever conventions, one can very well construct intuitive one-dimensional diagrams. 
E.g., the following string expresses the fact that the stretch of road between Park Avenue/35th Street and 
Park Avenue/36th is two-way, whereas that between Park Avenue/36th and Park Avenue/37th is one-way and 
proceeds from right to left: 

Park/35th <==> Park/36th <== Park/37th 

Owing to their representational power, diagrams are extensively used in a very wide range of fields. To note 
just a few examples, witness free-body, energy-level and Feynman diagrams in physics (Veltman 1995), arrow 
diagrams in algebra and category theory (Pierce 1991), Euler and Venn diagrams in set theory, function graphs 
in calculus and analysis, planar figures in geometry, bar-, chart- and pie-graphs in economics, circuit, state and 
timing diagrams in hardware design (Johnson, Barwise and Allwein 1996), UML diagrams in software design 
(Rumbaugh, Jacobson and Booch 1999), higraphs in specification (Harel 1988), visual programming languages 
(Chang 1990) and visual logic and specification languages (Agusti, Puigsegur and Robertson 1998, Hirakawa, 
Tanaka and Ichikawa 1990, Ogawa and Tanaka 2000), transition graphs in model checking (Berard, Bidoit, 
Finkel, Laroussinie, Petit, Petrucci and Schnoebelen 2001), ER-diagrams and hypergraphs in databases (Fagin, 
Mendeizon and Ullman 1982), semantic networks in AI, graphical user interfaces (GUIs) such as Xerox Parc's 
"Magic Lenses" (Bier, Stone, Pier, Buxton and DeRose 1993), and so on. As the capability of computers to 
store and manipulate diagrams improves, their use is likely to increase. 

Diagrams are not without drawbacks. While they often excel in depicting particular, concrete objects or 
situations, they are usually not as good for describing general, abstract structures and relationships. Roughly, 
the smaller and more concrete the class of models captured by a diagram, the more successful the diagram 
is likely to be. Spatial constraints tend to pull diagrams toward over-specificity, and end up limiting their 
generality and expressiveness as a result. To take an extreme example, diagrams cannot express tautological or 
contradictory information, lj 

Expressive limitations can lead to incorrect inferences. It is known that Euler circles (Euler 1768), for 
instance, are unsound. This follows from Helly's theorem in convex topology (Eggleston 1969). A simple 
illustration of the problem, due to Lemon and Pratt (Lemon and Pratt 1997), is the following: consider four 
sets A, B, C, and D, any three of which have non-empty intersections: 

An sn C*^0; 
AnenD^fb. 

These are three perfectly consistent premises. But any Euler diagram that tried to depict them graphically would 
lead to the incorrect conclusion that all four sets have a non-empty intersection (i.e. that AOBOCDD^ 
0), which does not follow from the premises. This is a consequence of a special case of Helly's theorem, 
which states that if any three out of four convex regions have a non-empty intersection then all four must have 
a non-empty intersection. Similar negative results hold for other diagrammatic ways of depicting sets and 
relationships between them, such as Englebretsen's linear diagrams (Englebretsen 1992); see Lemon's article 
(Lemon 2002) for a thorough discussion. 



3 Pierce diagrams can be viewed as a counterexample, but those rely on so many ad hoc conventions that they cannot be said to be 
analogical representations. 



The complexity of diagrammatic reasoning is another issue. Roughly, there are two types of diagrammatic 
inference. In one of them, exemplified by Euler circles and Venn diagrams, inference is carried out by drawing 
appropriate diagrams and then reading off the appropriate bits of information from the constructed picture. 
This type of diagrammatic inference is summarized by the slogan "If you can draw it, it holds."[jIn the second 
type of diagrammatic inference, exemplified in systems such as Hyperproof and in our own Vivid, inference is 
carried out in a more traditional sense, by deriving new diagrams from diagrams that are given as "premises," 
or by extracting sentential information from given diagrams. Computational complexity issues have been 
rigorously investigated for the former, but not for the latter. E.g., for the former, it has been realized that results 
obtained in studying the complexity of topological inference (Grigni, Papadias and Papadimitriou 1995) have 
a direct bearing on the complexity of drawing diagrams such as Euler circles, and hence on the first type of 
diagrammatic reasoning. For instance, it has been shown that propositional reasoning with Euler sets is NP- 
hard, even though reasoning about the same domain can be done polynomially using other representations 
(Lemon 2002). In the present work, it will be seen that even though MT>C proofs (Arkoudas n.d.a) can be 
checked for soundness in 0(n log n) time in the worst case (where n is the size of the proof), checking Vivid 
proofs can take exponential time, although it should be noted that in our case most of the complexity derives 
from dealing with unknown (incomplete) information. It would appear, therefore, that visual inference, at least 
in some cases, can be significantly more expensive than corresponding sentential reasoningP] 

For these and other reasons, researchers have concluded that logical reasoning frameworks must be het- 
erogenerous or hybrid (Barwise and Etchemendy 1995a, Myers 1994): they must support both diagrammatic 
and sentential modes of representation and reasoning, allowing users to freely combine the two. In the attempt 
to formulate a generic framework for heterogeneous reasoning, one naturally confronts the question of what 
type of diagrams to use. As Barwise and Etchemendy correctly observe (Barwise and Etchemendy 1995a), it 
would be impossible to construct a domain-independent framework for diagrammatic reasoning that relied on a 
specific type of diagrams. What makes a class of diagrams appropriate — i.e., good analogical representations — 
for certain problems might make them inappropriate for others. In the example of Barwise and Etchemendy, 
at different times electrical engineers use state diagrams, circuit diagrams, and timing diagrams to represent 
and reason about hardware as needed by the appropriate viewpoint at hand (control, logic gates, or timing, 
respectively). There is no single type of diagram that is uniformly appropriate. 

Nevertheless, we observe that much of what we do when we reason with or about diagrams does not depend 
on how diagrams are drawn or even on what they mean. In this paper we identify what is common in a great 
variety of instances of diagrammatic reasoning, and proceed to factor it out and extrapolate it into general 
principles. In the resulting framework, the type of diagrams used may vary from application to application, 
but the principles by which we reason with and about diagrams remain the same. This is not unlike other 
separations that are familiar from traditional, sentential logic: our vocabulary might vary from application to 
application (we have different constant, relation, and function symbols as dictated by the problem domain), and 
the interpetation of the atomic formulas that we can build from that vocabulary will also vary, but the general 
principles by which we reason with such formulas do not change. 



4 For instance, to check the validity of a syllogism with a Venn diagram, all we have to do is draw a figure that represents the premises 
of the syllogism. When done, the picture itself will tell us whether or not the conclusion follows; nothing further needs to be done. Hence, 
inference in such cases stops with the representation of the premises. In customary reasoning, by contrast, inference only begins after the 
premises have been represented. This is related to the notion of free rides (Shimojima 1996) in diagrammatic reasoning. 

5 There are alternative viewpoints, however. AI researchers have put forth the notion of vivid knowledge bases (Etherington, Borgida, 
Brachman and Kautz 1989, Levesque 1989), in which deductive retrieval can be performed particularly efficiently. Such knowledge bases 
consist only of ground sentences, ground inequalities, and universal quantifications. Etherington et al. (Etherington et al. 1989) claim that 
"the notion of vivid representations ... corresponds well to the kind of information expressed in pictures" and that "much of the information 
we gain (i.e., perceptually) occurs naturally in vivid form." Likewise, Levesque (Levesque 1989) states that "perhaps the main source of 
vividly represented knowledge is pictorial information." If that is indeed the case, one would expect pictorial reasoning to be efficient. 
Lemon (Lemon 2002), however, argues that such claims fail to take into account the type of spatial constraints that limit the expressiveness 
of diagrammatic representations. 



1.2 Notation 

For any sets A and B, A\B denotes the set-theoretic difference of A and B: 

A\B = {xeA\x<^B}. 

We write (a; b) for the ordered pair that has a and & as its first and second component, respectively. For any 
n > objects Xi, . . . , x„, [xi • • • x„] is the list that has x, as its i element. Given a list L = [xi • • • x„] and 
i € {1, • • • , n}, we write L(i) to denote x». Further, for any such i and object x, we define 

Pos(x,L) — {i G {1, . . . ,n} | x — Xi}. 

Accordingly, if x does not occur in L then Pos(x, L) = 0. If A is a set, then A* is the set of all lists of elements 
of A 

The empty list [] is a sublist of every list; no non-empty list is a sublist of []; while a list of the form 
L = [xi X2 • • • x n ] is a sublist of a list of the form L' — [yi j/2 • • • 2/m] iff (1) %\ — Vi an d [x2 • • ■ x n ] is a 
sublist of [y 2 • • • 2/m]; or (2) xi 7^ yi and L is a sublist of [y 2 • • • Vm)- 

For any set A, we write Poo (A) for the set of all finite subsets of A, When n is a positive integer, A n 
denotes the cartesian product 



Ax ■■ ■ x A, 

i.e., the set of all lists of length n with elements drawn from Ar\ Given a (partial) function / : A^ B and 
elements x € A, y 6 B, f[x h-> y] denotes that function from A to B which maps x to y and agrees with / on 
every other x' G A, More precisely: 

f (/\{(x;/(x))})U{(x;j/)} if / is defined for x; 
J[ x ^y\ J / u {( x; y )} otherwise. 

For 4'ci, /fl' denotes the restriction of i 7, on A', i.e., 

/M' = {(z;y)|/(x) = y and x e A'}. 

Finally, for an arbitrary relation R C Ai x ■ ■ ■ x A n , Z?(-R) denotes the set {Ai, . . . , A n }. 

1.3 Attribute structures and systems 

Definition 1: An attribute structure is a pair A — ({A\, . . . , Ak}; TV) consisting of a finite collection of 
sets Ai,...,Ak called attributes; and a countable collection 7Z of computable relations, with D(R) C 
{A u ..., A k } for each RgII. m 

An attribute structure is thus a type of regular heterogeneous algebraic structure (Meinke and Tucker 1992, 
Wechler 1992) (without any operators) whose carriers are called "attributes" for reasons that will become clear 
soon. We will tacitly assume that 1Z includes the identity relation on each attribute Ac {(a; a) \ a £ Ai}. 

We assume that there is a unique label li attached to each attribute Ai of a structure A. A label will serve 
as an alias for the corresponding attribute. Further, when the relations of A are immaterial, we identify A with 
its attributes. We can then write A simply as /1 : A\, . . . , l^ : Ak, where U is the label of A4, The number of 
attributes k is the cardinality of A, denoted by \A\. We say that A is finite iff every attribute of A is finite. 

6 With A 1 = A. 



Definition 2: Let A be any attribute structure. An attribute system based on A, or ^4-system for short, is a 
pair 

S = ({si,... ,s n };A) 

consisting of a finite number n > of objects S\, . . . ,s n and A. An attribute of A may include some (or all) 
of the objects S\, . . . , s n . If that is the case, S is called automorphic. We refer to the product n • \A\ as the 
system's power. ■ 

When A is obvious from the context or immaterial, we drop references to it and speak simply of "systems" 
rather than ",4-systems." 

Example 1: Consider a system consisting of a clock c, with two attributes, hours and minutes: 

({c}; hours : {0, . . . , 23}, minutes : {0, . . . , 59}). 

Another system based on the same attribute structure might consist of two clocks c\ and c 2 , perhaps indicating 
New York and Tokyo times, respectively: 

({ci, c 2 }; hours : {0, . . . , 23}, minutes : {0, . . . ,59}). ■ 

Example 2: Consider a system comprising the nodes of a three-element linked list, each with two attributes, 
a data field consisting of a Boolean value (t or f) and a next field consisting of another node or the null value: 

{{n\, n 2 , n 3 }; data : Bool, next : {m, n 2 , n 3 , null}), 

where Bool = {t, f} and null is a special token distinct from {m, n 2 , n%}. This is an automorphic system. ■ 

Example 3: Consider a blocks-world system consisting of three blocks A, B, and C, and a single "position" 
attribute, where a position is either a block or the floor: 

({A,B,C};pos : {A,B,C,floor}); 

and floor is distinct from A, B, and C. This system is also automorphic. ■ 

Example 4: Consider a Hyperproof (Barwise and Etchemendy 1995b) system consisting of four blocks and 
three attributes: a pair of integers (i, j) with < i, j < 9 indicating a grid location; a size (small, medium, or 
large); and a shape (cube, tetrahedron, or dodecahedron): 

({61, &2, &3, ^4}; loc : {1, . . . , 8} 2 , size : {small, medium, large} , shape : {cube, tet, dodec}) ■ 

Definition 3: A state of a system S = ({si, ■ • ■ , s n }, {A 1: . . . ,A^}) is a set of functions a = {S 1 , . . . ,6k}, 
where each <5j is a function from {si, . . . , s n } to the set of all non-empty finite subsets of Ai, i.e., 

8 i :{s 1 ,...,s n }^V 00 {A i )\<D. 

We refer to each Si as the state's ascription into Ai. An ascription 5i is a valuation if it maps every object to 
a singleton, i.e., if |<5i(Sj)| = 1 for every j = 1, . . . ,n. We may thus view a valuation as mapping every object 
to a unique attribute value. A world w is a state in which every ascription is a valuation. ■ 



t • >- f • - t • lh 

n\ n 2 n 3 

Figure 1 . 1 : A linked list world. 
A system that is based on a finite attribute structure has 

k 
TT 2 (|^ 4 |-1)» = 2 (Eti(l^l-l)") 
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states, where n is the number of objects and k the number of attributes, [j To simplify notation, when 5 is a 
valuation that maps an object s to a singleton {a}, we write S(s) — a instead of S(s) — {a}. Further, we 
will often use the label ^ of an attribute Ai to denote the corresponding ascription into Aj. That is, we are 
overloading the label symbols: sometimes U will stand for the attribute Ai and sometimes, in the context of 
a given state, it will stand for Si, the state's (unique) ascription into Af, the context will always make our 
intentions clear. As an additional convention, given a state a of the form described in Definition[3] an attribute 
(label) li and an object Sj, we write a(li, Sj) for Si(sj), i.e., the value of the ascription Si for the object Sj. 

Our notion of systems and states is similar to the corresponding notions in the model checking field (Clarke, 
Grumberg and Peled 1999, Berard et al. 2001), where a system is represented by a collection of variables and a 
state of a system is modeled by an assignment of a value (drawn from an appropriate domain) to each variable. 

Example 5: Consider the single-clock system of Example!!] 

({c}; hours : {0, . . . , 23}, minutes : {0, . . . , 59}). 

A state u\ of this system is given by the following two valuations: 

a i : hours(c) = lb,minutes(c) — 47, 

indicating a time of 3:47 p.m. This is a particular world of the clock system. Using the aforementioned 
convention, we can also write: 

o\(hours, c) = 15, ai(minutes, c) = 47. 

Suppose we know that it is between 2:30 and 3 past midnight, but do not know exactly how many minutes 
past 2:30 it is. This state of knowledge can be captured by the following state: 

<72 : hours(c) = 2,minutes(c) = {31, . . . ,59}. 
This state can also be expressed by writing 

cr2(hours, c) = 2, o~<i(minutes, c) = {31, . . . , 59}. 
Complete lack of information about the time is represented by the state: 

hours = {0, . . . , 23}, minutes(c) = {0, . . . , 59}. 



7 Our term "system state" corresponds roughly to what Barwise et al. (Barwise and Etchemendy 1995b) refer to as "situation." Our 
notion is much more general, as will be seen. 



Example 6: Consider the linked-list system of Example [2] The state 

data(n\) = t,data(n2) — i,data(n^) = t,next(n\) — ni,next(ri2) — n3,next(ns) = null 

depicts the world shown in Figure [T"T| The state 

data{jii) — {t, f}, data{ri2) = {t, f}, data(n^) — f,next(ni) = ni,next(n2) — {711,113}, next (n^) = null 

depicts a system in which we do not know the data fields of the first and second nodes, we know that the next 
field of the second node is either m or ns, and we have fixed values for the remaining nodes and attributes. ■ 

Example 7: Consider the blocks world system of Example [5] The state 

pos(A) — B,pos(B) — floor, pos(C) —floor 

depicts the blocks world shown in Figure [L2| The state 

pos(A) = {A,B,C,floor},pos{B) = {A, B,C, floor}, pos{C) = {A,B,C,floor} 

signifies complete lack of information about the positions of the blocks. ■ 

Example 8: Consider the Hyperproof system of Example |4] The state 
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should be self-explanatory at this point. ■ 

We might think of system states as mental models of situations, representing various states of knowledge 
ranging from completely specific to completely general. 

Definition 4: Consider a system S = ({si, . . . , s n }; l\ : A\, . . . ,h ■ Ak). We say that a state a' of S is 
an extension of another such state a, written a' Q a, iff a'{U, Sj) C a(U, Sj) for every i — 1, . . . , k and 
j = 1, . . . , ?i. [j We call a' a proper extension of a, denoted a' c a, iff a' C a and a %& '. ■ 

Hence, a' is a proper extension of a iff a' C. a and there is at least one attribute I and object s such that 
a' (I, s) C cr(l, s). Worlds do not have any proper extensions. 
Consider, for instance, the system of Example [T] 

({ci, C2}; hours : {0, . . . , 23}, minutes : {0, . . . ,59}). 

The state 

hours (ci) = {13, 14}, 

minutes' (ci) = {55}, (1.1) 

hours (02) = {6, 7}, 

minutes \c%) — {9, 10}, 



The terminology sounds somewhat paradoxical, since an extension of a state is one that assigns fewer attribute values to each system 
object, thereby making our knowledge of the system more specific. This is similar to the terminology of object-oriented class hierarchies, 
where we say that "human" is an extension of "mammal" to mean that the former is in fact a subset of the latter. 
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Figure 1.2: A blocks world. 



is an extension of the state 



hours( Cl ) = {13,14,15}, 

minutes(ci) = {55}, 

hours{c%) — {6,7}, 

minutesipi) — {9,10,11}. 



(1.2) 



The set of all states of S is arranged into a rich partial order corresponding to the join (union) semi -lattice 

(Voo(A 1 )\<l>)x---x(V 00 (A k )\^. 

We do not have a lattice because the meet of two states might not exist. This is related to the proviso of 
Definition [3] that ascriptions must map system objects to non-empty sets of attribute values, and ultimately 
stems from the expressive limitations of pictures. Given that diagrammatic ambiguity is part and parcel of our 
system, a join operator U on diagrams is fairly natural: for any attribute I and object s, we set 

{a i U cr 2 )(7, s) = <ii{l, s) U a 2 (l, s). 

This is precisely the least upper bound of the two states w.r.t. the ordering C. But a meet operator l~l would 
indicate conjunction, and conjoining diagrams with contradictory information is not pictorially meaningful. 
For instance, if an object s has a round shape in diagram o\ and a square shape in 172: 

ai(shape,s) — round 
a2(shape,s) — square 

then what is the shape of s is o\ n CT2? If we were to define meets as 

(<ji n <t 2 )(I, s) = 0-1 (7, s) n a 2 (l, s) 

then we would have o\ V\ a2(shape, s) — 0, an impossible state of affairs. This is why some researchers 
have argued that diagrams are essentially an impoverished form of sentential representations (Sober 1976). 
Sententially, we can very well construct a formula that asserts 

shape(s) = round A shape(s) = square 

but, diagrammatically, we cannot draw a square circle. This, in turn, is due to the fact that negation is not 
diagrammatically meaningful. If we had a negation operator — on diagrams then conjunction could be defined 
simply as a x l~l <r 2 = — (cti U ct 2 )- But negating a diagram could of course take us to the empty set if the starting 
value comprised the entire attribute space. 



If we admitted a special "null diagram" indicating an inconsistent state, then we could define complemen- 
tation and indeed joins and meets on diagrams, and we would obtain not just a lattice but a Boolean algebra 
isomorphic to sentential logic. Indeed, a mapping M from states to first-order sentences can be defined in 
a straightforward way, assuming we have chosen some binary predicate symbol A\ for each attribute I and 
appropriate constant symbols for the attribute values and system objects. We set: 



M(a) = f\ 



A V A ^a) 
leLaei(si) 



where s\ , . . . , s n are the system objects and L the set of all labels. E.g., the conjunction of the four senteces 

[hours(ci, 13) V hours{c\ 1 14)] A minutes(ci, 55) 
[hours(c2, 6) V hours(c,2, 7))] A \minutes{pz, 9) V minutesic.%, 10)) 

would correspond to the state ( |1.1[ ). The mapping M would then be a homomorphism: 

M(-ct) = -.M(<r) 
M Oi U ct 2 ) = M(ai) V M(cr 2 ) 
M(<ri n ct 2 ) = M(<7i) A M{a 2 ) 

1.4 Interpreting first-order languages into system states 

Consider a first-order vocabulary £ = (C, R, V) consisting of a set of constant symbols C; a set of relation sym- 
bols R, with each R £ R having a unique positive arity; and a set of variables V. An attribute intepretation 
of E into an attribute structure A = ({l\ : Ai, . . . ,1% '■ A k }; TV) is a mapping / that assigns, to each relation 
symbol R £ R of arity n: 

1. a relation R 1 £TZof some arity m, called the realization of R: 

R 1 C A il x • ■ ■ x A 4m 
(where we might have m ^ n); and 

2. a list of m pairs 

called the profile of R and denoted by Prof(R), with 1 < j x < n for each x = 1, . . . , m. 

As will become apparent soon, an attribute interpretation differs from a normal interpretation in that atomic 
formulas over system objects are "compiled" via profiles into atomic formulas over selected attribute values of 
(some of) those objects. Accordingly, an atomic statement concerning system objects must be understood as 
an atomic statement concerning certain attribute values of those objects. 
In what follows, fix a signature S = (C, R, V), an attribute structure 

A=({h :A 1 ,...,l k :A k };TZ) 

and an attribute intepretation / of £ into A. 



Suppose now that we are given an ,4-system S — ({si, . . . , s n }; A). We define a constant assignment 
as a partial function p from C to {si, . . . , s„}; while a variable assignment is a total function \ from V to 
{si, . . . , s n }- We write Dom(p) for the domain of a constant assignment p, i.e., the set of all and only those 
constant symbols for which p is defined. A total constant assignment will usually be written as p, with the hat 
indicating that the mapping is total. We will say that two constant assignments p 1 and p 2 have a conflict iff 
there is some c E Dom(p 1 ) n Dom(p 2 ) such that p 1 (c) ^ Pi{c)- Therefore, if Dom(p 1 ) D Dom(p 2 ) then p x 
and p 2 have a conflict iff p x 2 Pi- 

Formulas F over £ are defined as usual, with a term t being either a variable or a constant symbol. We 
omit definitions of standard notions such as free variable occurrences, alphabetic equivalence, etc. The set of 
variables that occur free in a formula F is denoted by FV(F). We regard alphabetically equivalent formulas as 
identical. A sentence is a formula without any free variable occurrences. For any term t, we define t p,x as p(c) 
if t is a constant symbol c and as x( v ) if t is a variable v. Since p is a partial function, t p,x may be undefined. 

By a named state we will mean a pair (er; p) consisting of a state er and a constant assignment p. We 
say that a named state (a 1 ; p') is an extension of a named state (a; p), written (a 1 ; p') C (er; p), iff er' is an 
extension of er (i.e., er' C er) and p' D p (viewing the partial functions p and // as sets of ordered pairs). Note 
that C. is covariant on the state components but contravariant on the constant assignments. We say that (er'; p') 
is a proper extension of (er; p), written (er'; p') \Z (a; p), iff (er'; p') C (er; p) and either a ' rz er or p' D p. 
Further, (er'; p') is a finite extension of (er; p) iff (er'; p') C (er; p) and the difference p' \ p is finite. We write 

(c'; /o') C (er; p) (or (er'; //) C (ct; p)) to indicate that (a'; p') is a finite extension (respectively, a finite proper 
extension) of (er; p). A named state (er; p) will be called a world iff er is a world (every ascription of er is a 
valuation) and p is total, n As before, worlds do not have any extensions. If (er'; p') C (er; p) we might say 
that (c';p') is obtainable from (cr;p) by thinning, or conversely, that (<r; p) is obtainable from (a';p') by 
widening. By an assumption base j3 we will mean a finite set of formulas. A context is a pair 7 = (/?; (er; p)) 
consisting of an assumption base j3 and a named state (er; p). Note that since the identity relation on each 
attribute is required to be decidable (by the computability proviso of Definition [TJ, the relation C is decidable 
as well. 

Lemma 1: The relation C is a quasi-order on named states, i.e., it is reflexive and transitive. 

We will now show how to assign a truth value — or an "unknown" token — to any formula F, given a named 
state (er; p) (of an ,4-system S — ({si, . . . , s n }; .4)) along with a variable assignment \. This is done by 
formally definining a mapping I/ a . p \, y from the set of all formulas to the three-element set 

{true, false, unknown} 

as follows. 

First consider an atomic formula R(ti, . . . ,t n ), where R is a relation symbol of arity n and profile 
[(k 1 ,ji),---,{ l i m ,jm)]- We set 

r true ifVa 1 £l n (tP*)---Va m £k m (t P '*).R 1 {a 1 ,...,a m ); 

7 (ff;p)/X^ tlj •-•'*»)) = \ false ifVa 1 £h 1 (t p ' i x )---Va m ek m (t P 'X).^R I (a 1 ,...,a m y, (13) 

[ unknown otherwise. 

In the first two cases above we tacitly assume — in the interest of readability — that t^' x is defined for every 
x = I, . . . ,m. If not, then the value of L a . p \ /~.(R(ti, . . . ,t n )) is unknown. 



Note that the occurrences of the symbol V on the right-hand side of ( 1 .3 1 occur as part of our metalanguage 
and should not be confused with object-level occurrences of V in Vivid formulas. We will continue to use 

'This also overloads the term "world": sometimes it refers to a state and sometimes to a named state. Again, the context will always 
disambiguate the use. 
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object-level symbols in different capacities without explicitly calling attention to the distinction; the context 
will always clarify the use. 

Sentential combinations of formulas are interpreted according to the strong three-valued Kleene scheme 
(Kleene 1952). For instance: 

( true if I {a;p)/x (Fi) = true and / ((j;p)/x (F 2 ) = true; 

j (<x;p)/x( FiAF2 )H false if/ ( CT ;p)/x( Fl ) = falseor/ (<x;p)/x( i?2 ) = false; (L4) 

I unknown otherwise. 

Finally, quantified formulas are evaluated as follows: 

{true if 7( ff . p ) /x[v „ Si] (F) = true for every i 6 {1, . . . , n}; 

false if J ((T;/o)/xbMSi] (F)= false for some iG {l,...,n}; (1.5) 

unknown otherwise. 

and 

{true if I {a;p)/ X {o>-*s i }i F ) = true for some i G {l,...,n}; 

false if I {a . p ) /x[v ^ Si] (F) = false for every ie {l,...,n}; (1.6) 

unknown otherwise. 

The following result is proved by a straightforward induction on the structure of F. It is the three-valued- 
logic version of the standard coincidence theorem of universal algebra and logic, which states that two variable 
assignments that agree on the free variables of a formula F are indistinguishable for the purposes of determin- 
ing the truth value of F. 

Lemma 2: If Xi( v ) = X2( v ) for every variable v that has a free occurrence in F, then 

/ (^;p)/x 1 ( i? ) = / (^;p)/x 2 ( F )- 
Lemma 3 (No unknowns in worlds): For every world (w;p), variable assignment \, and formula F, 

I {v ) ;p)/ X ^ F ) ^unknown. 
I.e., in a world every formula is either true of false. 

Proof: A straightforward induction on the structure of F. ■ 

Lemma 4 (Thinning preserves truth values): If(a';p') Q (a; p) and 

J (^;p)/x( F )^ unknown 
t^V ;p0/x (^) = / ((j;p ) /x (F). 

PROOF: By structural induction on F. For the basis case, suppose that F is an atomic formula R(t\, ■ ■ ■ , t n ), 



where R has a profile [(l^ , ji), . . . , (li m , j m )]- From ( 1.3 1, the assumption 

/ Kp)/x( F )^ unknown 
entails that the terms i?' x ..... t p ^ x are all defined, and further, that either 



V ai € l h (t p h x ) ■ ■ ■ V a m e h m {t P *) • R T (a 1: ..., a m ) (1.7) 
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or 

V Ql e U^ p ; x ) ■■•Va m e M*^) ■ "^Vi: ■ • •,«„,)• (1.8) 

Since <£'•*, . . . , t p * are all defined and p' is a superset of p, the terms if- ' x , . . . , t^^ are also defined. Further, 
since a' C. a, we have 

and since 

j.p'.x _ +p.x +p'.x _ +p.x 

we have 

ll 1 (t^)ci ll (t^),...J' m (t^)ck m (t%). 

Hence it follows that if (jTT7j» is the case then 

v«i e ij x (#*) -Vo m e C(*£ x ) ■ fl/ («i. • ■ • .«™). ( L9 > 



and therefore 7V CT /. P ')/ X (F) = Ii a . P )/ X (F) = true; while, if ( 1.8 1 is the case, we have 

Vai e l' il (t£ x )---Va m e C(*ilf) ■ ^?Vi,- ■ ■ ,a m ), (1.10) 

and therefore I/ a r. p ty (F) = I/ a . p y (F) = false. The inductive cases are straightforward. ■ 

Example 9: Consider the signature Si = (Q. c locki R c / oc fe, \l clock) where the set of constant symbols is 

Q/oc/t = {ci,c 2 ,. ■ •} 
the set of variables is \/ c i oc k — {xi, x 2 , . . .}, and the set of relation symbols is 

Rdock = { pM > AM ; Ahead, Behind}, 

with PM, AM unary and Ahead, Behind binary. 
Consider now the attribute structure 

Clock — (hours : {0, . . . , 23}, minutes : {0, . . . , 59}; {Ri, -R 2 , R3, -R4}), 

where R\ C hours, i? 2 Q hours, 

i? 3 C hours x minutes x hours x minutes, 

i?4 C hours x minutes x hours x minutes, 

defined as follows: i?i(/i) <^/i > 11, i? 2 (/i) <^/i < 11, 

R 3 (hi,mi,h2,m 2 ) <&hi > h 2 V (/ii = /i 2 A mi > m 2 ), 

and 

Ri(hi,mi, /i 2 , m 2 ) O-Zii < /i 2 V (/ii = /i 2 A mi < m 2 ). 

We define an interpretation / of Si into this attribute structure by specifying a unique relation (in the struc- 
ture) and a unique profile for each symbol in R c i oc k- In particular, we set PM 7 = R\, AM 7 = i? 2 , Ahead 7 = 
i?3, Behind 7 = -R4 and: 

Pro/(PM) = [(/wwra, 1)]; 

Prof (PM) = [(hours, 1)]', 

Prof (Ahead) — [(hours, 1), (minutes,!), (hours, 2), (minutes, 2)]; 

Prof(Behind) = [(hours, I), (minutes, 1), (hours, 2), (minutes, 2)]. ■ 

12 



Example 10: Consider the system ({ci, C2}; Clock), where Clock is the attribute structure of Example [9] Let 
a be the following state of this system: 

hours(ci) — {9,13}, 

minutes(c\) = 12, 

hours(c-i) = 8, 

minutes(c2) — 27, 

and let p be the partial constant assignment that maps ci to ci and C2 to c 2 . We claim that the sentence 
Ahead(ci, C2) is true in (a; p) for any variable assignment \. Indeed, consider an arbitrary \- Recalling the 
profile of Ahead, definition \\3) tells us that in order to have 

/ ( CT ;p)/ x ( Ahead ( c i' c 2)) =true 

we must have R(ai, 02, 03, 04) for all 

ai G hours{ci X = C\) = {9, 13}, 

a,2 G minutes(ci' x = c\) = {12}, 

03 G hours{c2 X = C2) = {8}, 

04 G minutes(c2 X = C2) — {27}, 

i.e., we must have R^(9, 12, 8, 27) and i? 3 (13, 12, 8, 27). Both of these hold according to the definition of i? 3 , 
since 9 > 8 and 13 > 8. 

As another example, the sentence PM(ci) A -iPM(ci) evaluates to unknown in (a; p), despite being patently 
inconsistent, because, intuitively, in (a; p)we do not know whether ci is prior to midnight or after it (one pos- 
sibility is 9, which is a.m., and the other is 13, which is p.m.). Therefore, PM(ci) evaluates to unknown, hence 
-iPM(ci) also evaluates to unknown, and therefore their conjunction evaluates to unknown as well. It is in- 
structive to see why, precisely, PM(ci) evaluates to unknown. Recall that the interpretation of PM is the unary 
relation R\, which holds of a given hour h iff h > 11; and that the profile of PM is [(hours, 1)]. Accordingly, 
for PM(ci) to be true in (a; p) (and arbitrary %), we must have Ri(ai) for all 

ai G hours(c p { x ~ c\) = {9, 13}, 

i.e., we must have 9 > 11 and 13 > 11, which is clearly false. Likewise, for PM(ci) to come out false in (cr; p) 
and \, we must have ^i? x (9) and ^^(13), which is also false. Accordingly, 

/ (^;p)/x( PM ( c i)) = unknown 



by (1.3» 



The following is a direct consequence of the finite size of the ascription values, the finite number of system 
objects, and Lemma|2] 

Lemma 5: li^. p y is computable for any named state (a; p) and variable assignment \- 
Definition 5: A world (w; p) satisfies a formula F w.r.t. a variable assignment x iff 

I (w;p)/ x ( F ) = t ™e- 
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We denote this by writing (w; p) \= x F. Likewise, we say that a world (w; p) satisfies a named state (er; p) 
iff (w; p) C (cr; p). This is denoted by (w; p) \= (a; p). We say that (w; p) satisfies a context 7 = (/3; (c; p)) 
w.r.t. a given x, written (w; p) ^ x 7, iff (w; p) |= x F for every F £ (3 and (w; p) |= (cr; p). Finally, we say that 
a context 7 entails a formula F, written 7 |= F, iff (w; p) |= x 7 implies (id; p) |= x F for all worlds («;; p) 
and variable assignments x- Likewise, 7 entails a named state (a'; p'), written 7 |= (cr'; p'), iff, for all worlds 
(w; p) and variable assignments x, we have (w; p) \= (cr'; p') whenever (w; p) \= x 7. ■ 

Lemma 6 (Weakening): If (/3; (a; p)) \= F then (/3 U /3'; (a; p)) \= F; and if (/?; (cr; p)) |= O'; /°') &en 
(/3U/3';(cr;p))hK;p')- 

Lemma 7: ff(# (a; p)) (= (a'; p') and (/?; (cr'; p')) |= F then (0; (cr; p)) |= F. 
PROOF: Pick any world (w; p) and variable assignment \ and suppose that 

(w;p)^ x ((3;(<j;p)). (1.11) 

Then, by the assumption (/3; (a; p)) |= (a 1 ; p'), we conclude 

(«;;£) M"V)- d-12) 

From ( |1.11[ ) and ( |1.12[ ) we infer 

(w;/3)ht(#("V))- d-13) 

Finally, ( |1.13| and the assumption (/3; (cr'; p')) |= F imply (w; p) |= x F. ■ 

Lemma 8: (/3;(cr;p)) |= (cr;p). 

Lemma 9: (/3 U {false}; (cr; p)) |= (<>"'; p')- 

Proof: Pick any world (u>; p) and variable assignment \, and assume 

(w;p)h x (/3U {false}; (cr;p)), 

so that (w; p) 1=^ false. But, by definition, 

/ (i U ;p)/ X ( false ) =false ' 

and the contradiction entitles us to infer (w; p) ^ (cr'; p'). ■ 

Lemma 10: If (# (a; p)) |= (a'; p') and (cr' ; p') C (cr"; p") then (/3; (a; p)) |= (a"; p"). 
PROOF: Pick any world (u>; p) and variable assignment \ and suppose that 

( W ;p)h x (/3;(^;p)), d-14) 

so that 

( W ;p)C(cr;p) (1.15) 

and 

W)/x(^)= true (U6) 



for all F € /3. From the assumption (/3; (cr;p)) ^= (c';p') and (1.14i we obtain (w;p) \= (cr';p'), which 
is to say (w; p) C (cr'; p'). Finally, (w; p) C [a 1 ; p'), the assumption (cr'; p') C (cr"; p") and Lemma [I] yield 

(u;;p)E(a";p"),i.e.,( W ;p)hK;p")- ■ 
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Corollary 11 (Widening is sound): If (a; p) C (</; p') then (/3; (ct; p)) |= (</; p')- 

PROOF: By Lemma[8] (/?; (a; p)) |= (cr; p), hence, by Lemma [To] (/?; (a; p)) \= (a'; p'). ■ 

Next we formalize the important notion of alternative extensions. 

Definition 6: Let o\ , <T2 be proper extensions of a state a. We say that o^ is an alternative extension of a 
with respect to a±, written Alt(a, a±, o-i), iff there is an attribute I and an object s such that: 

1. ai(l,s) c a(l,s); 

2. 0-2(1, s) — a(l,s) \ ai(l,s); and 

3. for all attributes I' and objects s', if I' ^ I or s' 7^ s then <T2(l', s') = a(l' , a'). ■ 

It follows immediately that if such an atribute and object exist, they must be unique. 

As a simple example, consider a system consisting of one object s with two attributes, color and size, and 
suppose that a stipulates red, green, and blue as the possible color values of s and large, medium and small 
as its possible size values; and suppose that o\ extends a by limiting the color values of s to green and blue 
and its size to small: 



color(s) 




color (s) 




size(s) 



size'(s) 



What counts as an alternative extension of a w.r.t. o\ ? Considering that o\ essentially states that the color of s 
is either green or blue and that its size is small, we could differ from it in one of the following respects: 



Color of s 


Size of s 


{red} 


{large, medium} 


{red} 


{small} 


{green, blue} 


{large, medium} 


{red} 


{large, medium} 



That is, we could either choose to (1) disagree with the color, and either disagree or agree with the size (the 
latter choice is immaterial in light of the first disagreement), resulting in the top two rows of the table above, or 
(2) disagree with the size and either agree or disagree with the color (again, this being immaterial), which leads 
to the third and fourth rows. Given that set memberhip represents disjunctive information, we can collapse the 
first two and last two possibilities, obtaining: 



Color of s 


Size of s 


{red} 


{small, large, medium} 


{red, green, blue} 


{large, medium} 
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These are the only two alternative extensions of a w.r.t. a\. In general, given an arbitrary extension <j\ C a, 
we can effectively construct all alternative extensions of a w.r.t. o\. There are m such extensions, where m is 
the number of attribute-object pairs (or a.o. pairs for short) (I; s) such that a±(l, s) ^ a (I, s), or equivalently, 
such that <7i(Z, s) C a (I, s); i.e., the number of pairs of attributes and objects whose corresponding ascription 
values changed in going from a to a\ . We can generate the alternative states by taking the complement of the 
ascription value of each such pair in o\ Fj (clause 2 of Definition rob while reverting the other m — 1 pairs to 
their a values (clause 3 of Definition [6]). 

We stress that in determining the alternative extensions of a w.r.t. <j\ we only consider those objects and 
attributes that are changed by <j\ . We ignore those ascription assignments that remain the same in going from 
a to (T\. As another example, there are two states that are alternative extensions of ( 1.2 1 w.r.t. (fTTTh. In one of 
them we keep the original hours values of ci ({13, 14, 15}) but complement the minutes value of C2 to obtain 
{11}; while in the other alternative we keep the original minutes value of c^ ({9, 10, 11}) but complement the 
hours value of C\ to obtain {15}. In both cases the minutes of C\ and hours of c-i remain the same as in the 
original state fll.2) , as neither of them was modified by ( |l.l| i. 

Lemma 12: If w, cr' C cr and w%a' then there is some a" C cr such that Alt(cr, cr' , cr") and w C. a" . In 
words: if a world w and a state a' both extend a and w does not extend a', then there is an alternative 
extension a" of a w.r.t. a' such that w extends a". 

Proof: Since both a' and w are extensions of a, we have 

a'(l,s)Ca(l,s) (1.17) 

and 

w(l,s) Ccr(7,s) (1.18) 

for every attribute I and system object s. Further, since w % a', there exist an attribute k and an object Sj such 
that w(k, Sj) 2 c'ihi Sj), i.e., there is some attribute value a such that 



and 



a G w(li, Sj) 



a g a'{k,Sj). 



Moreover, since w is a world we have w(U, Sj) = {a}. From ( 1.19i and ( 1.18 1 we infer 



a £ a(k,Sj). 
From ( fl~2l"T ), ( fL20| ), and ( (LT7) we obtain 

a'(k,Sj) C <j(k,Sj). 

Now define a" c a as follows: 

a"(k, Sj) = <r(k,Sj) \ a'(k, Sj) 

while for every attribute / and object s such that I ^ l L or s ^ Sj, set 

a"(l,s) = <j{l,s). 



(1.19) 

(1.20) 

(1.21) 

(1.22) 
(1.23) 

(1.24) 



It follows by construction (specifically, from (1.221, (1.23i, and (1.24i) that Alt(a, a 1 , a"). Further, w C. a" . 
To see this, consider any attribute I and object s. Either I = li and s = Sj, or not. In the former case we have 
w(l, s) = {a}, so from ( 1.23 I, ( 1.21 1, and ( 1.20l we conclude a € u"(l, s), hence w(l, s) C a"(l, s). In the 
latter case, w(l, s) C a" {I, s) follows from (|1.24|i and O}. ■ 



The complement with respect to the corresponding ascription value in a. 



16 



We now generalize the foregoing notion of alternative extensions so that it obtains w.r.t. to several states instead 
of just one. We will see that the new definition (Definition[9]below) subsumes the one given above. 



Definition 7: A list of to > 1 a.o. pairs [(li; s%) ■ ■ ■ (l r , 
■ ■ ■ = s m , i.e., iff all to pairs are identical. 



j)] is homogeneous iff l\ = • • • = l m and si = 



Definition 8: Let <ri, . . . , a m C <7, m > 1. A list of to a.o. pairs L — [(7i; si) • • • (l m ; s m )] spans the states 
(7i , . . . , a m with respect to a iff 

o-i(k,Si) C a(k,Si) 



for every i = 1 , . 

[«i • • • w] of [1 



, m. In addition, we say that L properly spans ai, . . . , er m w.r.t. er iff for every sublist 
to] such that \L(i\) ■ ■ ■ L(i m /)] is homogeneous we have 






C cr^j^SiJ. 



Equivalently, L does not properly span ai , . . . , cr m with respect to a iff for some such sublist we have 

cr^ih^SiJ U • • • U ai m , (li m , , s im , ) = o-^jSjJ. ■ 

Note that every list of length one that spans a\ w.r.t. a (for <j\ C cr) does so properly. That is why the definition 
below is a proper generalization of Definition [6] 

Definition 9: Let a%,. . .,a m ,a' C <r, to > 1. We say that cr' is an alternative extension of a w.r.t. 

<7i, . . . , <7 m , written Alt(a, {a±, . . . , a m }, a'), iff there is a list i = [(?i; si) • • • (l m ; s m )] properly spanning 
(7i, ... , cr m w.r.t. ct such that for every attribute I and object s we have 

a'(l,s) = a(l,s)\ |J er 4 (J,«). 

iePos((l; s), X) 

We write A({cri, . . . , <r m }, a) for the set of all alternative extensions of a w.r.t. <j\, . . . , cr m . ■ 

Therefore, to compute all alternative extensions of a w.r.t. <j\ , . . . , a m we need to compute all lists of a.o. pairs 
that properly span o\ , . . . , a m w.r.t. a. We will present algorithms for both tasks shortly, but we first turn to an 
example that will help to clarify these definitions. 

Example 11: Suppose we have two objects s\ and S2, and two attributes, color and size, with color having 
three possible values: red, green and blue (abbreviated R, G, and B); and where size has three possible values: 
small, medium, and large (ab. S, M, and L). Suppose further that the starting state a is as follows: 



Color( Sl ) = {R,B} 



Size(s 1 ) = {S,M,L} 
Color{s 2 ) = {R,B,G} 



Size(s 2 ) = {M, L} 



Now consider the following three proper extensions of a: 
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o\ 


o 2 


0-3 


Color(s x ) = {B} A 


Color( Sl ) = {R,B} 


Color(si) = {i?} F 


Size(s 1 ) = {S,M} B 


Size(sx) = {£} D 


Sfee( Sl ) = {S,M,L} 


Color(s 2 ) = {B,G} C 


Color(s 2 ) = {R, B, G} 


Color(s 2 ) = {R, B, G} 


Size(s 2 ) = {M, L} 


Size(s 2 ) = {L} E 


Size(s 2 ) = {M,L} 



We have used the labels A — F to mark those a.o. pairs (I; s) for which state Oi properly extends a, i.e., such 
that cri(l, s) C a(l, s). The following lists of a.o. pairs span a\, <r 2 , and 0-3 w.r.t. a: 

L\ = [(Color; si) (Size; Si) (Color; si)] (corresponding to A-D-F) 



L 2 = [(Color; s\) (Size; s 2 ) (Color, si)] 
L 3 = [(Size; Si) (Size;si) (Color; si)] 
L4 = [(Size; si) (Size;s 2 ) (Color; si)] 
L5 = [(Color; s 2 ) (Size; Si) (Color; si)] 
Lq = [(Color; s 2 ) (Size;s 2 ) (Color; si)] 



(A-E-F) 
(B-D-F) 
(B-E-F) 
(CDF) 

(C-E-F) 



These are the only lists that span <7i, a 2 , and 0-3 w.r.t. a. From these, only L4, L5, and Lq do so properly. L\ 
does not span <7i, o 2 , and 0-3 properly (w.r.t. a) because [1 3] is a sublist of [1 2 3] such that [£i(l) Li(3)], 
namely [(Color; S\) (Color; Si)], is homogeneous and yet 

ai(Color, s\) U ^(Color, si) — {R, B} (jL a(Color, s\) — {R, B}. 
L 2 fails for the same reason. For L3, [1 2] is a sublist of [1 2 3] such that 

[L 3 (l) L 3 (2)] = [(Size; Sl ) (Size; Sl )} 
is homogeneous but 

a\(Size, Si) U a 2 (Size, s\) — {S, M, L} <f_ a(Size, si) = {S, M, L}. 

Accordingly, we have a total of three alternative extensions of a w.r.t. <7i, a 2 , and 03, corresponding to L4, L5, 
and Lq: 



a 4 (B-E-F) 


a 5 (C-D-F) 


a 6 (C-E-F) 


Color( Sl ) = {B,G} 


Color( Sl ) = {B,G} 


Color( Sl ) = {B,G} 


Size(si) = {L} 


Size(si) = {S,M} 


Size(s 1 )^{S 1 M 1 L} 


Color(s 2 ) = {R,B,G} 


Color(s 2 ) = {R} 


Color(s 2 ) = {R} 


Size(s 2 ) = {M} 


Size(s 2 ) = {M,L} 


Size(s 2 ) = {M} 



so that 



A({<7i, cr 2 , ct 3 }, cr) = {cr 4 , er 5 , cr 6 }. 



Thus each alternative state is uniquely determined by the corresponding list that properly spans ui, a 2 , and 0-3 
w.r.t. cr. Specifically, the ascription values of each alternative state are obtained by "flipping" (complementing) 
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the corresponding ascription values of a on the relevant coordinates of the respective spanning list. For coor- 
dinates (a.o. pairs) (I; s) that are not in the spanning list L, the original values of a are retained, since in those 
cases we have Pos((l; s),L) = and hence 

< J ( l ,s)\ (J <Ti(l,s)=(j(l,s). 

iePos((l; s), L) 
Intuitively, this ensures that every alternative state has a maximal disagreement with each extension of a. ■ 
The following algorithm computes the set of lists that properly span states <j\, . . . , a m w.r.t. a: 

1. Let $ be the set of all a.o. pairs for the system at hand. The size of this set will equal the power of the 

system. 

2. Let IP be the set obtained from 3> m by filtering out all those lists [(7 X ; S\) ■ ■ ■ (l m ; s m )] for which 

3 i g {1, . . . ,m} . (T l (l i ,s i ) = a(k,Si). 

That is, 

# = {[(h; Si) ■ ■ ■ (l m ; s m )} e $ m | <Ti{k,Si) C a{k,Si) fori = l,...,m}. 

Thus ^ is the set of all and only those lists that span <j\ , . . . , a m w.r.t. a. 

3. From ^, filter out all those lists that do not properly span ci, . . . , a m w.r.t. er, and return the result. To 
determine whether a list \(l\\ s\) ■ ■ ■ (l m ; s m )] in ^P properly spans a\, . . . , <j m w.r.t. a, do the following: 

• Let / be a function that maps a.o. pairs to sets of positive integers. Initially, set / <— Xp . for any 
a.o. pair p. 

• Let P <- 0. 

• For i = 1, . . . , to: 

/<-/[&;*) -»/(Ji,*i)U{t}]; 

P^PUJfeSi)}. 

• For each pair (7; s) e P: if 

(J cri(;,s) = cr 
ie/((/;s)) 
return /afoe, else continue. 

• Return true. 

With this algorithm, we can easily compute A({<7i, . . . , <J m } 1 cr) as follows: 

1. Let ^ be the set of all and only those lists of a.o. pairs that properly span a\, . . . , a m w.r.t. a. 

2. Let E <- 0. 

3. For each list Leif: 

• Let a' be the unique state such that for any / and s, 

a'(l,s) = a(l,s)\ |J <Ti(l,8). 

iePos((l; s), L) 
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• S^SU{cr'}. 

4. ReturnS. 

The reader will verify that the more general definition of alternative state extensions subsumes the former 
notion in the following sense: 

Lemma 13: Alt{a, a', a") iffAlt(a, {a'}, a"). 

The following result generalizes Lemma [T2| 

Lemma 14: If a\, . . . , a m , w \Z a and w%<Ji for every i = 1, . . . , m, then there is some a' C o such that 
Alt(a, {f7i, . . . , <r m },a') andw C u' . 

Proof: By assumption, we have 

Vl,s.w(l,s) Ca(l, s ); (1.25) 

Vie {l,...,m}.Vl,s.<Xi(l,s) Ca(l,s). (1.26) 

Further, for each i = 1, . . . , m there is some a.o. pair (k;Si) such that 

w(k,Si) % (Ti(li,Si), 

meaning that there is some attribute value a, such that 

w(k, Si) = {on} (1.27) 

and 

a l (£a i (l ll s i ). (1.28) 

From \\.21) and ( |1.25| ) we infer 

Vie{l,...,m}.ai€o-(Zi,Si). (1.29) 

Hence, from ( |1.28| l and ( |1.26| l we conclude 

V i e {1,. . . ,ra} . <Ji{li,Si) c a(k,Si). (1.30) 

Therefore, the list 

L = \{li\si) ■ ■ ■ (l m ;s m )} 

spans (Ji, ... , <j m w.r.t. a. Moreover, it does so properly. To see this, consider any sublist [i\- ■ ■ i m >] of 
[1 • • • m] such that [L(ii) ■ ■ ■ L(i m /)] is homogeneous, so that 

and hence 

w{l il ,S il ) = • •• = w(l im ,,Si m ,), 

which is to say 

a tl = ■ ■ ■ = oa m ,. (1-31) 

Now suppose, by way of contradiction, that 

m 

U Vijik^Sij) = aik^SiJ. (1.32) 
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From ( |1.27[ ) and ( |1.25[ ) we conclude 



so that 



Hence, by (1.32[>, 



w(^ i; s n ) = {a n } C CT(/ il ,s il ), 

Oil e c(iii,Sti)< 



3=1 



(1.33) 



which means that there is some j £ {1, . . . , to'} such that 



(1.34) 



Fro m (|1.28| l we get o^ ^ cr 4j (/j j ., s^.). But, by ( |1.31[ ), a H = a^, hence a il £ o"^(Z^,s^), contradict- 
ing ( 1.34 1. Therefore, L spans <7i, . . . , a m w.r.t. a properly. 
Now define a' C a as follows: for any I and s, 



ieftw((i;a),L) 



(1.35) 



By construction, Alt(a, {<ti, . . . , <r m }, a'). Further, we have w C. a'. To prove this, we need to show that 
w(l, s) C a' (I, s) for all I and s. To that end, consider arbitrary I and s. Either (I; s) occurs in L or not. If not, 



then Pos((l; s),L) — and hence, from (1.35 I, a' (I, s) = a(l, s), so w(l, s) C a' (I, s) follows from ( 1.25 i 



Suppose, by contrast, that (I; s) occurs in L, so that 

Pos((l;s),L) = {h,...,im'} 
for some m! such that 1 < ml < to. From ( |1.28| l, 

Vj £ {l,...,rri} . a tj g <T l] (k j ,s l] ). 
But 



(1.36) 



and since 

we get a h 



Oi I = w(k 1 ,Si 1 ), ■ ■ ■ ,(Xi m , = w{k m , 1 s im ,), 

Accordingly, dl. 36b yields 

Vj e {!,..., to'} . ttjj ^cr.Ak Si ), 



(1.37) 



which, by virtue of ( 1.37 1, becomes 



It follows from ( p~38] > that 



Vj £ {!,..., m'} . Otj ^ a^Q, s). 



n & IJ cr 4j (Z,s), 
i=i 



(1.38) 
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or, equivalently, 

a h £ \J ai{l,s). (1.39) 

iePos{{l; s), L) 

However, 

a»! € <j{k x ,8 ix ) = cr(l,s), (1-40) 

and hence we infer from ( |1.39| l, ( |1.40| i, and ( |1.35| ) that 

a n ea'{l,s). (1.41) 



But, from ( 1.37 1, wQ^^s^) = w(l, s), which is to say w{l, s) — {a^}. We have thus shown that, in this case 



too, w(l,s) C a'(l,s). ■ 

We now extend the notion of alternative extensions to named states. 

oo 

Definition 10: Let {ay, p x ), . . . , (a m ;p m ), {a'; p') c {a; p), m > 1. We say that {a'; p') is an alternative 
extension of {a; p) w.r.t. {ay, Pi), • ■ • , (cm! p m ), written 

Alt{{a; p), {{ay p x ), ..., {a m ; p m )}, (a'; p')), 

iff Dom(p') = Dom(p 1 ) U • • • U Dom(p m ) and there is a subset S 1 C {1, . . . , m} such that: 

1. p' conflicts with p t iff i E S; and 

2. if 5 ^ {1,..., m} then A/?(cr,{o-j | ie {l,...,m}\S},cr / ). ■ 

Owing to the first condition, if such a subset S 1 C {1, . . . , to} exists then it is unique. When to = 1 we might 
write Alt((a; p), (en; p x ), (a'; p 1 )) instead of Alt((a; p), {(ar, p,)}, (a'; (/)). 

The following algorithm computes all alternative extensions of (a; p) w.r.t. {ay, p x ), . . . , (a m ;p m ): 

1. Let p\, . . . , p' k , k > 1, be all and only the constant assignments on Dom(p 1 ) U • • ■ U Dom{p m ) that are 
supersets of p. Note that there are k — n d such assignments, where n is the number of system objects 
and 

d= \[Dom{p 1 )U ■ ■ ■ U Dom{p m )}\ Dom{p)\. 

2. Leti? = 0. 

3. For each p\, % = 1, . . . , k, do the following: 

• Let Sj C {a i , . . . , a m } consist of all and only those states &j,j G { 1 , . . . , m} such that p\ does 
not have a conflict with p , meaning that p\ D p . 

• Let$ i = A(E i , ( r). 

• Seti?^i?U{(cr';pO | ct' G$J. 

4. Return i?. 

The algorithm is rather naive in that it may duplicate some work in the process of computing A(Si, a) for the 
various i. Memorizing intermediate results and building A(Sj, cr) incrementally could improve its efficiency. 

oo 

Lemma 15: If {ayp x ), . . . , (a m ;p m ) C (a;p),m> 1, (w;p) C (a;p),and 

Vie {l,...,m} . (>;p) g (cr 4 ;pj) 

then there is {a' ;p') \z (a; p) such that Alt{{a; p),{{ay, p x ), . . . ,{a m ; p m )},(cr; p)) and (iu;p) C (a';p r ). 
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Proof: The following holds by assumption: 

V* € {1, . . . , to} . w % (Ji V p 2 Pi 



Define 
and let 
so that 
It follows by construction that 

and 



S = {iE{l,...,m} \p2Pi} 

p' = p\ [Dom{p 1 ) U • • • U Dom(p m )], 

P^p'^P- 

Dom(p') = Dom(p 1 ) U • • • U Dom(p m ) 



(1.42) 

(1.43) 
(1.44) 
(1.45) 



Vi € {l,...,m} . p' 2 Pi^i € S, 

which is to say that p' has a conflict with p i iff i £ S. At this point there are two cases: S = {1, . . . , to} or 
S C {1, . . . , to}. In the first case, we must have 



p'^P, 



(1.46) 



for if p' — p then, from ( 1 .44 1, p 1 



Pr, 



p and hence p D p t for all * = 1 , . . . , to, since p C /?by 



assumption. But, from ( 1.43 I, \/i £ {1, . . . , to} . pD p i would entail S = , cont radicting the supposition S 



{1, . . . , to} (recall that to > 1). Define a' — w. Then (a'; p') C (a; p) by (1.45 i, and indeed (a'; p') E (a; p) 
by ( 1.46 1 and (1.44 1. In addition, (w; p) C (cr'; p') follows from w C w and ( 1.45 1. 



By contrast, suppose that S C {1, . . . , to}, so that 

{1,...,to}\5^0. 
From the definition of S 1 and ( |1.42| ) we infer 

\/ i £ {I, . . . ,m}\ S . w % (Ti. 

From ( |1.48[ ) we can infer that 

Vie{l,...,TO}\S , .a,Ccr, (1.49) 

for otherwise there would be some j £ {!,..., to} \ S such that <Tj \/_ a and <Tj C a, and hence <Tj = a. But 



(1.47) 



(1.48) 



w C cr = <Tj contradicts the assumption w % <jj. Further, we can infer w \Z cr, for, in light of ( 1 .47 1, w = a 



would contradict ( 1.49 1, given that worlds do not have any proper extensions. Therefore, by Lemma 
exists a state 

cr' IZ (7 



14 there 



such that Alt(a, {cr* | i £ {1, . . . , to} \ S}, a') and 



w C a' . 



(1.50) 



(1.51) 



From ( 1.50 1 and ( 1.45 1 we conclude (cr'; p') iz (a; p). Further, by construction, 

Alt((a; p), {(ai, Pl ), . . . , (a m ; p m )}, (a'; //)), 



while (w; p) C (a'; p') follows from (|1.51| and ( 1.45 1. This concludes the case analysis. 
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Corollary 16: If (a'; p') \Z (a; p), {w; p) Q (cr; p), and (w; p) % (a'; p') then there is some 

K;p")c(<r;p) 
suchthatAlt({a;p),{a';f/),((r";p")) and (w;p) Q (a";p"). 
We end this section by introducing the following notion of state entailment: 

Definition 11: Suppose that (<ji; p-J, . . . , (cr m ; p m ) C (cr; p) and let (3 be any assumption base. We say that 
(cr; p) entails (<n; p x ), ..., (a m ;p m ) w.r.t. /3, written (cr; p) llt-73 {(en.; p^, . . . , (cr m ; p m )}, iff for every (a'; p') 
such that 

Alt((a; p), {(o-i; Pi), . . . , (<r TO ; p m )}, (cr'; p')) 

there is some F £ (3 such that, for all %, 

V;p')/x( F ) = false - ■ 

When n=lwe drop the braces and write (a; p) \\\-p (<n; Pi) instead of (a; p) llh^ {((Ji; Pi)}. 

This definition captures the intuition that any world which extends the state (cr; p) and satisfies the formulas 
in j3 must also extend one of the states (en; p^), in the sense that any alternative way of extending (cr; p) will 
end up falsifying some element of f3. (Of course if there are no alternative ways of extending (a; p) then the 
entailment holds vacuously, even if f3 — 0.) This is formally demonstrated by the proof of LemmafTTlbelow. 

Determining whether or not (a; p) llh^ {(en; pjj, . . . , (a m ; p m )} is decidable; we present an algorithm for 
it which makes use of an auxiliary function g that takes a formula F and a named state (a; p) and returns true 
or false. To compute g(F, (a; p)): 

1. Let t/>i, . . . ,tpk be all distinct functions from FV(F) to the set of system objects {si, . . . , s n }. (There 
are k — n) "{ )\ such functions.) 

2. Let Xi, • ■ ■ , Xfc be arbitrary variable assignments such that 

3. If Ii a . p)/ x . — false for every i = 1, . . . , k then return true, else return false. 

The algorithm for determining (a; p) WVp {(cti; p ± ), . . . , (cr TO ; p m )} can now be stated thus: 

1. For each (a 1 ; p') such that Alt((cr; p), {(ai; pj), . . . , (cr m ; p m )}, (cr'; p')): 

• If 3 F € /? . 5(-F 1 , (cr'; p')) then continue, else return false. 

2. Return true. 

The algorithm clearly hinges on g, whose correctness in this context depends on Lemma[2] 

Lemma 17: If (a; p) llh^ {(01; p ± ), . . . , (cr m ; p m )} then for all worlds (w;p) and variable assignments \, if 

(w,p) \= x (0;(a;p)) 
there is some i £ {1, . . . , m} such that (w;p) |= (en; pj. 
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Proof: Assuming 

(er; p) Wrp {{v\.',Pi), ■■■, (c m ; p m )} 

pick any world (w; p) and variable assignment \ an d suppose that (w; p) \= x (/?; (a; p)) so that 

(w;p) C (er;p) 

and 

VFe/3.J (w . ?)/x (F)=true. 

By way of contradiction, suppose that there is no i E {1, . . . , to} such that (u>; p) |= (o^; p,), i.e., 

Vi e {1,...,to} . (w;p) % (a l ;p l ). 



By Lemma 15 there is some 



such that 
and 



(a';p') C (<r;p) 

Alt((a; p), {(o-i; p^, • . . , (cr m ; p m )}, (er'; p')) 

(t»;p) C(cr';p'). 



(1.52) 

(1.53) 
(1.54) 



(1.55) 
(1.56) 

(1.57) 
(1.58) 



But then, by Definition 1 1 and ( 1 .55 1 it follows that there is some 

Gef3 

such that 

V;p')/x( G ) = false ' 
and hence, from the Thinning Lemma (LemmaHjl in tandem with ( 1.56 1 and ( 1.58 1, we obtain 

-W)/ x ( G ) = false > 

which contradicts ( |1.54[ ) in view of ( |1.57| i. ■ 

Corollary 18: If (a; p) \hp {a'; p 1 ) then (/3; {a- p)) \= (a'; p'). 

1.5 A family of diagrammatic natural deduction languages 

We now introduce Vivid, a family of natural deduction languages in the DPL tradition (Arkoudas 2000) that 
combine sentential and diagrammatic reasoning. A concrete instance of Vivid is obtained by specifying a 
vocabulary £ = (C, R, V), an attribute structure A = ({l\ : j4i, . . . , If. : A^}; TV), and an interpretation I of R 
into A. We assume in what follows that S, A, and / have been fixed. The terms and formulas of the language 



are defined as described in Section 1 .4 We write F [t/v] to denote the formula obtained from F by replacing 
every free occurrence of v by the term t (taking care to rename F if necessary to avoid variable capture). The 
following result is readily proved by induction on the structure of F. 



Lemma 19: Ifb e {true, false}, 

and v' does not occur in F then 



I (a;p)/ X [v^s]( F )- b 
hv, P)/Xiv' ^ s]( F [V* / v }) = b - 
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1.5.1 Abstract syntax 

There are two syntactic categories of proofs, sentential and diagrammatic. Sentential deductions are used to 
derive formulas, while diagrammatic deductions are used to derive diagrams. We will see that the two can be 
freely mixed, and indeed that their structures are mutually recursive. We use the letters D and A to range over 
sentential and diagrammatic deductions, respectively. The symbol S will range over the union of the two. The 
abstract syntax (Reynolds 1998) of both proof types is defined by the grammars below: 



D ::= RuleApp 

assume F D 
F by D 

pick-any x D 

pick-witness w for 3 x . F D 
specialize Vxi ■ ■ ■ x n . F with t\,. 
ex-generalize 3 x . F from t 
cases from F 1; ... ,F k : (a^pj) - 
observe F 






£>;A 

claim (a; p) 

(a; p) by thinning with F u . . . , F n 

(a; p) by widening 

(a; p) by absurdity 

cases from F±, ... ,F k : {<Ji]Pi) — ► Ai 

cases Fi V F 2 : F x -» Ai | F 2 -» A 2 

pick-witness w for 3 x . F A 



I 0n!/O 



D n 



I On!/?™) 



A T 



D 



DI A 



where the syntax of inference rule applications is as follows: 



RuleApp ::= claim F 
true-intro 

modus-ponens F => G, F 
modus-tollens F => G,^G 
double-negation -i -^F 
absurd F,^F 
left-andF A G 
right-andF A G 
both F, G 
left-either F, G 
right-either F, G 
cases Fi V F 2 ,Fi => G,F 2 
left-lffF & G 
right-iffF 4» G 
equivF =>• G,G => F 



G 
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The composition operator ";" associates to the right by default, so 2)i; D2; £>3 stands for 

£>i;(5>2;2>3) 

rather than (5)i; 2)2); £>3- Parentheses or begin-end pairs can be used to change the default grouping. 

We define 2) [t/x] as the deduction obtained from 5) by replacing every free occurrence of the variable x 
by the term t, taking care to perform a-conversion as necessary to avoid variable capture. The definition is 
given by structural recursion: 

(D 1 ;D 2 )[t/x] = 55i[t/x];:D 2 [t/x] 

((a; p) by thinning with F\ , . . . , F n ) [t/x] = (a; p) by thinning with F\ [t/x] , . . . ,F n [t/x] 

(cases from F 1 , . . . , Fk : _ cases from Fi [t/x] ,... ,Fk [t/x]: 

(a i;Pl ) -► Ai| ••• I (a n ;pj -► A n ) [t/x] (a i;Pl ) -► Ai [t/x] \ ••■ | (a„;p„) -► A„ [t/x] 



(cases Fi V F 2 : 
Fi -» Ai |F 2 -> A 2 )[t/x] 



cases Fi [t/cc] V F 2 [t/a;]: 
Fi [t/x] -» Aj [t/i] I F 2 [t/x] -» A 2 [t/x] 



(pick-witness a; for 3y . F A) [t/x] = pick-witness a; for (3 y . F) [t/x] A 

pick-witness w for (3 y . F) [t/x] A [t/x] 
pick-any x F/ 
pick-any y D [t/x] 



(pick-witness w for 3 y . F A) [t/x] 
(when x 7^ w) 

(pick-any x D) [t/x 



(pick-any y D) [t/x] 

(when x 7^ jy) 



We omit the defining equations for the sentential pick-witness, which is handled like the diagrammatic 
pick-witness; and for the remaining cases from, which is treated like the one above. The definition for the 
other forms is straightforward and can be found elsewhere (Arkoudas 2000). In all cases we assume that the 
deduction has been a-renamed away from the given term t. 

1.5.2 Evaluation semantics 

Our formal semantics is given by axioms and rules that establish judgments of the form 
and 



which are read as: 



7 h A ^ (c; /?) 



"In the context 7, deduction D (A) derives F (respectively, (<j; p))." 



The semantics of most sentential deductions are straightforward generalizations of the standard AfVC 
semantics (Arkoudas n.d.a). We illustrate here with the axiom for left-and and the rule for assume, omitting 
the rest: 



{/3 U {F A G}; (a; p)) h left-and F AG~~>F 
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\Thintiiti£~\ 

(p U {Fi, . . . , F n }; (a; p)) h {a'; p') by thinning with Fi, . . . , F„ ~> (V; p') 

provided (a; p) HI"{F 1 ,...,F n } ( CT ';p') 

[Widening] 



(/3; (<r; p)) h (<x'; />') by widening -> (a'; p') 
provided (it; p) C. (a 1 \ p') 



(fi U {false}; (<r; p)) h (<r'; p') by absurdity ~v (<r'; p') 

[Diagram-Reitaration] 

(/3; (<r; p)) h claim (<r; p) ~> (<r; p) 

(/3U{F 1 ,...,F fc };(<r 1 ;p 1 ))hA 1 ^( ( x' ; p') 
( / 3U{F 1 ,...,F fc };(<r„;p„))l-A„->( -';p') 



[Absurdity] 



(/3u{Fi,...,F fc };(ff;/9)) I- cases from F 1: ...,F k : (cn; Pl )-+ Ai | ••• | (<r„;pj -+ A n ~»(<T';p') 
provided (cr; p) IH- {Fl ,...,F fc } {(°i;/»i)> • ■ ■ , ( ff n;p„)} 



(/3 U {Ft V Fa, Ft}; (a; p)) h Ai -^ (a'; p') (/3 U {Fi V F 2 , F 2 |; (a; p)) h A 2 ~-> (a'; p') [C - 2 ] 
(/3 U {Fi V F 2 ; (<r; p)) h cases Fi V F 2 Fi -> Ai | F 2 -► A 2 ~v (<r'; p') 



W;(a;p))hD^F (f3U {F};(a; p)) h A^ (a'; p') [D A] 
(/3;( ff ;/,))hD;A-(«7';p') 



(f3;(*;p))hA^(<r';p') (f3;(*>; p>)) h D ^ F [A; D] 
(/3;(<r;p))l-A;D^F 



Q3;Q;p)) i- Ai^> (giiPjj (ff;Qi;Pi)) k a 2 ^» (q2\p?) [a ; A] 

(/3;(a;p))|-Ai;A 2 ~»(<T 2 ;p 2 ) 



(/3; (a; p)) hJi^Fi Q3 U {Fx}; (a; p)) h D 2 ^ F 2 [D . D] 
(/3;(a;p))|-Di;D 2 ->F 2 



(g U {3 x . F, F fs/ttl}; (a; p)) h A \z/w] ~ (a'; pf) [EI/A] 

(f3U {3x . F}; (a; p)) h pick-witness w for 3 x . F A ~~» (a'; p') 
provided z is fresh 



Figure 1.3: Formal semantics of diagrammatic deductions 
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[Ci 



(f3U{F};(a;p))hD^G 
(/?; (a; p)) h assume F D^ F^G 

The only new sentential forms are observe, cases from, and A; D. We will discuss the last two later; the 
semantics of observe are as follows: 

[Observe] 



(P;(a;p)) h observe F^ F 
provided that I/ a . P )/^{F) = true for all \ 

The side condition is computable because of Lemma Bland because, by Lemma 13] we need only be concerned 
with the free variables of F. In fact usually F is a sentence (it has no free variables) and hence we only need 
to consider one (arbitrary) variable assignment. 

We now turn to the semantics of the various Vivid constructs for case analysis. There are four types of 
case reasoning in Vivid: 

Sentential-to-sentential: In this type of reasoning we note that a disjunction F\ V F2 holds and that a formula 
G is entailed in either case. That entitles us to conclude G. This is captured syntactically as a rule 
application: 

cases F X \J F 2 ,F 1 ^>G,F 2 ^G. 

The semantics of such rule applications carry over from AfVC unchanged, since there is no diagram 
manipulation involved: 



{j3 U {Ft V F 2 , Fi ^G,F 2 ^ G}; (a; p)) h cases F t V F 2 ,F 1 ^G,F 2 ^G-^ G 

Sentential- to-diagrammatic: Here we note that a disjunction F\ V F 2 holds and proceed to show that a cer- 
tain diagram (a; p) follows in either case. This is captured by the syntax form 

cases Fi V F 2 : F x -> Ai | F 2 -> A 2 , 

which is classified as a diagrammatic deduction ("a A") since the end result is a diagram. The semantics 



of this form are given by rule [C 2 ], shown in Figure 1.3 



Diagrammatic-to-sentential: We note that on the basis of the present diagram and some formulas F\ , . . . , Fk 
in the assumption base, one of n > other system states (ci; p x ), . . ., (er„; p n ) must obtain, and proceed 
to show that a formula F can be derived in every one of these n cases. This entitles us to infer F, provided 
of course that the n diagrammatic cases are indeed exhaustive. This form of reasoning is captured by the 
form 

cases from F x ,...,F k : (criiPi) ~> D i I ' • • I K;P„) -* D n . 

This is classified as a sentential deduction, since the end result is a formula F, Its semantics are shown 



in Figure 1.4 The caveat that the diagrams (<n; p x ), . . . , (c„; p n ) form an exhaustive set of possibilities 



on the basis of F x , . . . , F/. and the current diagram is formally captured by the proviso 

(cr; p) \\\-{ Fl ,... tFh } {(o-i;pi), • • • , 0„; p n )}. 

Diagrammatic-to-diagrammatic: This is similar to the above mode of reasoning, with the exception that 
instead of deriving a formula F in each of the n cases, we derive a diagram. Therefore, syntactically, 
following each of the n cases we have diagrammatic deductions Ai, . . . , A„ (rather than sentential 

29 



(pU{F 1 ,...,F k };(a l ;p 1 ))hD 1 ^F 






((3U{F 1 ,... 1 F k };(a n ;p n ))^D n ^F 




[C 3 ] 


(/? U {F u . . . , F k }; (a; p)) h cases from F l ,...,F k : fa; p x ) - 


♦AI 


0„;p„)- 


>D n ^F 




provided fa p) \\\- {Flj ... tFk} {fa; p x ), ..., (a n ;p n )} 







Figure 1.4: Semantics of diagrammatic-to-sentential case reasoning. 

deductions Di, . . . , D n as we did above), and the entire form is classified as a diagrammatic deduction, 
since the final conclusion is a diagram. The following syntax form is used for such deductions: 

cases from F 1 ,...,F k : fajpj) -> Ai | ••• | fa;p n ) -> A„. 

The corresponding semantics are given by rule [Ci], shown in Figure [O] 

Likewise, there are four types of deduction sequencing: 

1. D\ ; D2, where a sentential deduction D\ is composed with another sentential deduction Di- This form is 
classified as a sentential deduction, since the end result is a formula (the conclusion of D 2 ). Its semantics 



are given by rule [D; D] of Figure 1.3 They are isomorphic to the regular composition semantics of 



AfVC, since there is no diagram manipulation involved. 

2. £>; A, where a sentential deduction D is composed with a diagrammatic deduction. This form is classi- 
fied as a diagrammatic deduction since the end result is a diagram — the conlusion of A. Its semantics are 
prescribed by rule [D; A]. Observe that the conclusion of D becomes available to A (e.g., the conclusion 
of D could be a disjunction and A might be a diagrammatic case analysis of that disjunction). 

3. A; D, where a diagrammatic deduction A is composed with a sentential deduction. This form is clas- 
sified as a sentential deduction since the end result is a formula (the conlusion of D). Its semantics are 
given by rule [A; D]. Conclusion threading here is also intuitive: D will be evaluated in the system state 
resulting from the evaluation of A. E.g., D might be an observe deduction that points out something 
that can be seen in the diagram derived by A. 

4. Ai; A2, where a diagrammatic deduction Ai is composed with another diagrammatic deduction A2. 
This form is of course classified as a diagrammatic deduction, since the end result is a diagram (the 
conlusion of A2). Its semantics are given by rule [A; A]. The same principle of conclusion threading 
applies here: A2 is evaluated in the system state resulting from the evaluation of Ai; the assumption 
base is threaded through unchanged. 

Theorem 20 (Soundness): If 7 h D ~^> F then 7 |= F; and if 7 h A ^ (a; p) then 7 (= (a; p). 

PROOF: We proceed by induction on derivation lengthFH We will omit most sentential forms, as those have 
been proved sound elsewhere (Arkoudas 2000). 



"To be perfectly precise, we are proving the statement: "For all positive integers n and for all 7, D, A, F, and (a; p), if there exists 
a derivation of length n of the judgment -j h D"^ F then ■y \= F; and if there exists a derivation of length n of 7 h D -v-> F then 
7 |= (<t; p). It is readily seen that this statement implies Theorem|20| 
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The basis cases correspond to the axioms of our semantics. In what follows we will treat the diagrammatic 
axioms [Observe], [Absurdity], [Diagram-Reitaration], [Widening], and [Thinning]. 

• [Observe]: In this case D is of the form observe F and we need to show that 

whenever (/?; (a; p)) h D ~> F. To that end, consider an arbitrary world (w; p) and variable assignment 
X and suppose that (w; p) \= x (/?; (a; p)), so that 

(w;p)Q(a;p). (1.59) 

By the side condition of [Observe], it must be that 

and hence, from ( |1.59| > and Lemma [4] 

J («;;p)/x( F )= true ' 

which is to say (w; p) |= x F. We have thus shown that (w; p) |=y (/3; (u; p)) implies (w; p) |=y F for 
any (w; p) and x, which establishes (/3; (ct; p)) |= F. 

• [Thinning]: Here A is of the form 

(a'; p') by thinning with F 1 ,...,F n 

and we need to show that if 

(13 U {F lt . . . , F„}; (a; p)) h A ~» (a'; p') (1.60) 

then 

G9U{Fi,...,JJ' n };(a;p))(=(er , ; / y). (1.61) 

From ( |1.60| l and the side condition of [Thinning] we obtain 

(cr;p) ll!--r^ 1 ,..., J F„} (cr'; /?'), 



and hence, by Corollary [18J ({-Fi, . . . , F n }; (a; p)) |= (cr'; p'). Now ( 1.61 1 follows from weakening 
(Lemma |6). 

• [Widening]: Here A is of the form 

(a'\ p') by widening 

and we must show that (/3; (a; p)) |= (a 1 ; p') whenever ((3; (a; p)) hA^ (a 1 ; p'). From the side con- 
dition of [Widening] we infer (a; p) C (cr'; p'), and now the desired (/?; (cr; p)) |= (cr'; p') follows from 
Corollary [TT] 

• [Diagram-Reitaration]: Here the result follows directly from Lemma[8] 

• [Absurdity]: Here A is of the form 

{cr';p') by absurdity 

and we need to show 

(/? U {false}; (cr;p))hK;p') 
whenever (/3 U {false}; (cr; p)) hA^ (a 1 ; p'). This follows from LemmaRj] 
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• [Ci]: Here A is of the form 

cases from F u ...,F k : (a^, Pl ) -> A x | • • • \(a n ;p n ) -> A n 
Consider any assumption base /3 and named states (er; p), (a'; p'), and assume that 

(0U{F 1 ,...,F k };(a;p))\-A<^(<r';p?). 

We need to show 

(PU{F 1 ,...,F k };(a;p))^(a';p'). 

From ( |1.62| > and [C\\ we infer 

V*€{l,...,n}.(/8U{Fi,...,F fc };(<7 i ;p,))l-A i -*((/;//) 

and 

(a; p) \\\- {Fll ..., Fk } {Ol5 Pi), ■ ■ ■ , (<r„; P n )}- 

Pick any world (tu; p) and variable assignment x and suppose that 

(w;p)\= x (0U{F l ,...,F k };(a;p)) 

so that 

(w;p)\= x ({F u ...,F k };(a;p)). 



From ( 1.65 1, LemmafTTl and (1.67b we conclude that (w; p) |= (cr^; p.) for some j e {1, 
the inductive hypothesis, (|1.64 1 yields 



and since 



(/3U{F 1 ,...,F k };(<j J ;p J ))^(a';p'), 
{w,p)\= x {0U{F u ...,F k };{<T j ;p j )), 



it follows from ( |1.68| that (w; p) |= (a';p'). 
[C2]: Here A is of the form 



and, assuming 
we need to show 



cases F 1 V F 2 F 1 ^A 1 \F 2 ^ A 2 
(/3U{FiVF 2 };(a;p))WA->(c/;p'), 



(/3U{FiVF 2 };(<r,p))h(^;p')- 
To that end, consider an arbitrary world (w;p) and variable assignment x such that 

(u;;p)hx(/?U{F 1 VF 2 };(a;p)) 

so that 

/ («,;p)/ X ( F l) = trUe 



(1.62) 
(1.63) 

(1.64) 
(1.65) 

(1.66) 

(1.67) 
, n}. By 

(1.68) 



or 



I { W ;-p)/ X ^) = true 



(1.69) 

(1.70) 

(1.71) 

(1.72) 
(1.73) 
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(note that this inference would not be sanctioned in a weak three-valued Kleene logic). Now from ( 1.69 1 
and [C 2 ] we get 

{ft U {F 1 V F 2 , F 1 }; {a; p)) hA^ (a'; pi) 

and 

{ft U {Fi V F 2 , F 2 }; {a; p)) h A 2 ~> (a'; p'). 



(1.74) 
(1.75) 



Inductively, ( 1.74 1 and ( U 5\ respectively yield 

{f3U{F 1 yF 2 ,F 1 };{a;p))\={</;p') 

and 

{/3U{F 1 VF 2 ,F 2 };{a;p))^{a';p'). 

Now if ( |1.72| i holds then, from (JTTTTJ, we have 

{w;p)^ x {/3U{F 1 VF 2 ,F 1 };{(T;p)), 
and hence {w; p) |= {a 1 ; p') follows from ( 1.76 1; while if ( 1.73 1 holds then 



(1.76) 
(1.77) 



{w;p) \= x {ftU {F^V F 2 ,F 2 };{a; p)), 
and hence {w; p) |= {a 1 ; p') follows from ( 1.77 1. Therefore, {w; p) |= {<j'\ p') holds in either 1 
• [C3]: Here A is of the form 

cases from F 1 ,...,F k : {01, p x ) -*■ D x \ ■■■ \ {a n ;p n ) -> D n . 
Pick any (3, F, and (er; p), and suppose that 

{0U{F 1 ,...,F k };{a;p))^A^F, 
so that 



Vi€{l,...,n}.{ftU{F 1 ,...,F k };{a i ;p i ))hD i ^F 



and 



(1.78) 

(1.79) 
(1.80) 



{a; p) \h{F u ... ,F k } {Oi; Pi), ■ ■ ■ , 0„; p„)}. 
We need to show (/3 U {Fi , . . . , F k }; {a; p) ) ^ -F. To that end, pick any (w; p) and x and assume that 

{w;'fl\= x {0U{F 1 ,...,F k };{*;p)). (1.81) 

It follows that 

{w;p)^ x {{F 1 ,...,F k };{a;p)), 



and hence by Lemma 17 and ( 1.80 1 we conclude that {w; p) C (<Tj-; p ) for some j <E {1, . . . , n}. Induc- 
tively, from ( |1.79| l, we infer 

{PU{F 1 ,...,F k };{a J ;p J ))\=F. (1.82) 



But from {w; p) C (o\,-; p ) and ( 1.81 1 we get 



{w;p)\= x {ftU{F 1 ,...,F k };{a i ;p i )), 



and therefore ( 1.82 1 yields (u>; p) |= x F, 
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• [Ell A]: In that case the deduction is of the form 

pick-witness w for 3 x . F A 

and assuming that 

(/3 U {3 x . F}; (a; p)) h pick-witness w for 3 x . F A ~~+ (a'; p'), 

we need to show 

{/3U{3x.F};{a;p))\=(a';p'). 

To that end, consider any (w; p) and \ sucn that 

(w,-p)^ x (/3U{3x.F};(a;p)). 
From ( |1.83| l and the [EI/A] rule we infer that, for some fresh variable z, 

08 U {3 x . F, J 1 [*/*]}; (a; p)) h A [«/«;] ~» (<r';p')- 



From ( 1.86 1 and the inductive hypothesis we obtain 

((3u{3x.F,F[z/x}};(a;p))\=(a';p'). 



From ( 1.85 1 and ( 1.6 1 we conclude that there is some system object s such that 

I (w;p)/ x [x^s]( F ) =true. 
Therefore, from Lemma p~9| 

I (w;p)/ X [z ^ s]( F [z/x]) = true ' 

and since z does not occur in j3 U {3 x . F}, we also have (by ( |1.85| l and Lemma[2|: 

V G e 0U {3 x . F} . I {wrp)/x[z M s] (G) = true. 

Hence, 

(w; p) h x[z _♦ S ] |8 U {3 x . F, F [z/x]}, 



and since (w;'p) Q (a; p) (from (1.85 i), we conclude that 

(w,p) h x [z ^ s] (f3U{3x.F,F [z/x]}; (a; p)). 
Finally, from \\&9\ and \\&1\ we obtain (w; p) \= (a'; p'). 
• [D; A]: Here the deduction is of the form D; A, and assuming that 

(P;(a;p))\-D;A^(a';p'), 
we need to show 



Pick any (w; p) and \ an d suppose that 



(/3;(a;p))hK;p')- 



( w ;p) hx(A(o-;p)). 



(1.83) 
(1.84) 

(1.85) 
(1.86) 
(1.87) 



(1.88) 



(1-89) 



(1.90) 

(1-91) 

(1.92) 
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From ( |1.90| l and the [D; A] rule we infer that, for some F, 

(ft(a;p))hD^>F, 

(/3U{F};( ( r;p))hA^(a';p')- 



(1-93) 
(1.94) 



From (1.93 i and the inductive hypothesis we obtain (/3; (a; p)) \= F, which, in tandem with (1.92i, 
yields 

Therefore, 



(w;p)\= x ((3U{F};(a;p)). 



Now (1.94 1 and the inductive hypothesis give 

(/3U{F};(a;p))hK;p'), 



(1-95) 
(1-96) 



and finally ( 1.95 i and ( 1.96 1 produce the desired (w; p) |= x (a'; p'). 
• [A; D]: Here the proof is of the form A; D and assuming that 



(1.97) 



we need to show (/?; (a; p)) |= F. Accordingly, consider any world (w; /5) and variable assignment \ 
such that 

(w;p)\= x ((3;(a;p)). (1.98) 



From ( 1.97 1 and the [A; D] rule we conclude that, for some (a'; p'), 

G9;(<r;/0)l-A~>(aV) 

and 

{!3;{a';p'))hD^F. 



(1.99) 
(1.100) 



From ( 1.99 1 and the inductive hypothesis we get (/3; (a; p)) \= (a'; p'), so ( 1.98 i yields 

(w;p)^(a';p') 
and hence 

( W ;?)hx(/9;K;p'))- 



(1.101) 



But from (1.1 00 » and the inductive hypothesis we get (/3; (er';//)) \= F, which, along with (1.101 1 
entails (w;p) \= x F. 



[A; A]: Here the deduction is of the form Ai; A2. Assuming 

(/3;(a;p))hA i; A 2 ^(a 2 ;p 2 ), 
we must show (/3; (a; p)) \= (cr 2 ; p 2 ). Pick anv ( w > P) an( l X sucn that 

O;?) \= x {0;{a;p)). 



From ( 1.102 1 and rule [A; A] we infer that, for some (<ti; /t^), 

(/3;(a;p))hA 1 ^(a 1 ;p 1 ); 
(/3;(^i;Pi)) 1- A 2 ~> (o- 2 ;p 2 ). 



(1.102) 

(1.103) 

(1.104) 
(1.105) 
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From ( 1. 104 1, ( 1. 105 I, and the inductive hypotheses we get 

(/?; (a; p)) (= {?i\p x ); 
(/3; {a x ; Pi)) h (0-2; p 2 )- 



(1.106) 
(1.107) 



From ( 1.103 1 and (1.106 1 we infer (w; p) |= (<ti; /c^), so that 



which in tandem with ( 1. 107 1 yields the desired (w; p) \= (02; p 2 )- 

This completes the case analysis and the induction. ■ 

Example 12: Consider the Vivid language obtained by fixing the clock signature, attribute structure and in- 
terpretation of Example[9] Now consider a system of two clocks c\ and C2, to which we will give the names ci 
and C2 (recall that ci and C2 are constant symbols of the signature, so this is a constant assignment p, which 
need only be partial). Now let a be the state depicted by the following picture: 



{4,5,6}:28 

Cl 



5:45 



C2 



Intuitively, this state signifies that we know the precise time displayed by c 2 (5:45 am). We are also sure of the 
minute value of c\ (28), but not of its hour value, which could be either 4, 5, or 6. Now suppose that we are 
further given the premise Ahead(ci, C2), indicating that the time displayed by c x is ahead of that displayed 
byc 2 . 

From these two pieces of information, one diagrammatic and the other sentential, we should be able to 
infer the following diagram, call it a': 



6:28 



5:45 



Cl 



c 2 



That is, we should be able to conclude the exact time of c x , since, given that c x is ahead of c 2 , the hour displayed 
by it cannot possibly be 4 or 5; it must, therefore, be 6. We can do this in Vivid with the following one-line 
proof: 

(cr';p) by thinning with Ahead(ci, C2). 

This deduction, when evaluated in the context ({Ahead(ci, C2)}; (a; p)), will result in the state (diagram) 
(cr'; p). More formally, we have the following judgment: 

({Ahead(ci, c 2 )}; (a; p)) h (a 1 ; p) by thinning with Ahead(ci, c 2 ) ~> (V; p) 



by virtue of 

O; P) llh {Ahead(C!,c 2 )} W\ P)- 
Note that p does not change in the resulting state. 



(1.108) 



To establish ( 1.108 1 rigorously, we must show that for all named states (er"; p") such that 

A^((<7;p)>';p)>";p")) 
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we have 

/( CT '' ; p'')/ x (Ahead(ci,c 2 )) = false 



for all variable assignments x> according to Definition 11 Given that the assignment p does not change 



it follows from Definition 10 that we must have p" = p and hence Alt(a,a',a"). Now there is only one 
alternative extension a" of a w.r.t. a', obtained from a by complementing the hours value of c\ in a' with 
respect to the corresponding value in a: 

a" : hours(ci) = {4,5}. 

It is straightforward to verify that 

/((//. p )/ x (Ahead(ci, c 2 )) = false 

for all x- ■ 

1.6 Representing arbitrary graphs 

Graphs (including trees, lists, etc.) are very widely used as diagrammatic depictions of structured data. In this 
section we present a way of modeling arbitrary graphs in our framework as system states. These ideas will be 
put to use in the example of Section [L7| 

Consider an arbitrary finite graph G = (N; E), where N is a set of nodes and E C N x N a set of 
directed edges. Typically we wish to attach a value to each node n G N, so we assume we have a function 
data : N — > V that maps each node to some element of a set of values V. For the purposes of drawing the 
graph, we also assume that the children of every node are ordered from left to right, i.e., we assume there is 
a function children : N — > N* (arbitrary lists can be chosen if the ordering is immaterial for displaying the 
graph). Consider, for instance, the graph: 




Here N = {ni, n 2 , n^} and E = {(ni, 122), (n\, 713)}. The values attached to the nodes are natural numbers. 
So we can represent the graph by the functions data and children as mentioned above, where 

data(ri\) = h,data(ji2) = i 1 data{n^) = 3 

and 

children{ni) = [n%, 713], children(n 2 ) — [], children{n^) = []. 

This is similar to the "adjacency list" representation of graphs (Cormen, Leiserson and Rivest 1990). 

Any graph G — (N; E) where the nodes take values from a set V gives rise to systems of the form 
Sn = {N; An), where An is an automorphic attribute structure of the form 

An — (id : N, children : N*,data : V;1Z). 

Here the attributes children and data are as discussed above, id is the identity function on N, and D(R) C 
{N, N* , V} for each relation R e 1Z (the precise contents of 1Z will vary). The graph G itself can be repre- 
sented as a world of the system S n- "Incomplete" graphs where the values and/or children of some nodes are 
not precisely known can be represented by partial states of such systems. 
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[5 8 3 2] 




[2 3 5 8] 



Figure 1.5: The call graph resulting from the application of Merge Sort to the list [5832]. 



1.7 Another example: the Mergesort puzzle 

In this section we present a more involved Vivid language by way of a puzzle. In its general form, the puzzle 
can be described as follows. The output of an algorithm is displayed at the bottom of a diagram depicting a call 
graph for a particular run of the algorithm. Some sentential information might also be given in addition to the 
diagram. The objective is to infer what input(s) could possibly have resulted in the given call graph, or, more 
precisely, what inputs are consistent with the given information (the call graph and the sentences). Inference 
is mostly performed diagrammatically, by deriving a sequence of successive call graphs, by performing case 
analyses involving such graphs, etc. It will be seen that such graphical proofs are considerably more compact 
and intuitive than sentential analogues. In the next section we illustrate the puzzle informally with Mergesort, 



while in Section 1.7.2 we formalize it rigorously as an instance of Vivid. 



1.7.1 Guessing the input of Mergesort 

Mergesort is a popular 0(n log n) sorting algorithm. The algorithm works according to the divide-and-conquer 
paradigm (Cormen et al. 1990): it successively halves the given list until the original input has been broken into 
one-element pieces, which are trivially sorted; this is the dividing phase. The small lists are then repeatedly 
combined into larger and larger sorted lists, until we finally obtain the correct sorted permutation of the original 
input. This is the conquering phase, which turns on the fact that once we have two sorted lists, say [2 8] and [1 
3 5], we can efficiently merge them to get another sorted list, in this case [12 3 5 8]. 



For example, Figure 1.5 depicts the call graph obtained by applying Mergesort to the input list [5832]. 
Note that the graph is a DAG (directed acyclic graph). Diverging edges on the top half represent recursive 
applications of Mergesort to the left and right halves of the input (dividing phase); while converging edges on 
the lower half represent calls to the merging procedure (conquering phase). We make the convention that when 
the input list is of an odd length 2n + 1, we take the first n elements as the left half and the remaining n + 1 
elements as the right half. 

The call graph for an application of Mergesort is completely and unambiguously determined once the input 
list is given. However, things are more interesting in the reverse direction. Clearly, there is no way of retrieving 
the input list from the output alone, since the inverse of a sorting function is a relation, not a function — any 
one of n\ initial permutations could result in the same sorted n-element list. But if, in addition to specifying 
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the output, we also constrain the call graph of the algorithm by sprinkling some tidbits of information on it or 
by specifying some sentential information along with it, then we may be able to infer the original input, or at 
least narrow it down to relatively few possibilities. 

As a simple example, suppose you are told that the output of Mergesort is [1 2 5 8]. At this point there 
is not much of interest you can conclude — there are 4! = 24 possible inputs that could produce this output. 



But suppose you are further told that the corresponding call graph is as shown in Figure 1 .6 where we have 



attached labels Ni to each node of the graph for easy reference. We write Ni = ? to indicate that we do 
not know anything about the list that should appear at node Ni', we write Ni D {x\, ...,Xk} to indicate 
that the numbers x\, . . . , Xu occur in the said list (though in unknown order, and possibly in tandem with 
other numbers); and Ni = [x\ ■ ■ ■ Xk] to indicate that we know the exact value of the list in question to be 
Xk]. From the diagram of Figure 1.6 along with what we know about Mergesort, we can conclude that 



\X\ 



the original input was either [2 5 8 1] or [5 2 8 1]. 

The proof consists of two parts: first we derive a sequence of six increasingly detailed diagrams from the 
initial diagram of Figure [L6] each extending the previous one, culminating with a diagram in which we know 



the exact values of all the lists except those for N lt N 2 , N4 and N 5 ; this part of the proof appears in Figure 1 .7 



We then perform an exhaustive case analysis by observing that there are only two possibilities at this point: the 
lists of A^ and N§ are (a) [2] and [5], respectively; or else they are (b) [5] and [2], respectively. In the first case 
we can deduce that the input list was [2 5 8 1], while in the second case we can deduce that it was [5 2 8 1]. 
Therefore, we can infer that the input list was either [2 5 8 1] or [5 2 8 1]. 

Let us analyze the proof in more detail, beginning with the first part shown in Figure [L7| That part consists 
of six steps, labeled (1) through (6). The new information extracted by each step appears in red for enhanced 
clarity. We discuss each step below: 

• Step (1) infers that Nq must contain the number 8. This follows because we know that 8 occurs in Ng 
but not in N-?; and that, since Nq and N? converge in Ng, a number can occur in Ng iff it occurs either 
in Nq or in A7 (this holds because converging edges indicate list merging). 

• Step (2) infers that the list appearing in node Nq must be precisely [8]. We already know from the 
previous step that 8 occurs in the said list. Now if the list had any additional elements, its length would 
be greater than one, and hence it would be longer than the N-? list, which we know to have only one 
element. But this cannot be the case because Nq and N? are the left and right halves of the JV3 list, and 



N4, 




[1] 



N w = [1258] 



Figure 1.6: A partially unknown MergeSort call graph resulting in the output [12 5 8]. 
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JVi = ? 

N 2 = ? 7V 3 = ? 



Ni = ? 

iV 2 = ? iV 3 = ? 







iV 4 = ? N 5 = ? N 6 = ? N T = [1] ==l iV 4 = ? JV 5 = ? AT B D {8} iV 7 = [1] =Q 



N a = ? iV 9 D {8} 



iVio = [1 2 5 ■ 



iVi = ? 



iVs 



iV 9 3 {8} 



iVio = [1258] 
JVi = ? 



iV 2 = ? % = ? 



iV 2 = ? % = ? 







iV 4 = ? iV 5 = ? iV 6 = [8] JV 7 = [1] ==l iV 4 = ? JV 5 = ? iV 6 = [8] N 7 = [1] ==> 



iV 8 = ? iV 9 D {8} 



ATio = [125, 



iVi 



iV 2 = ? iV 3 = [8 1] 



iV 8 = ? JV 9 = [1 8] 

iVio = [1258] 
TVi = ? 



iV 2 = 



N 3 = [8 1] 







iV 4 = ? N 5 = ? JV 6 = [8] JV r = [1] ==> iV 4 = ? iV 5 = ? iV 6 = [8] iV 7 = [1] 



N s 



N 9 = [1 8] 



iVio = [125. 



N a 3 {2, 5} 7V 9 = [1 8] 



Nio = [125 



Figure 1 .7: First part of a graphical proof solving an instance of the Mergesort puzzle. 
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JVi = ? 



N 2 = ? 



N s = [8 1] 





iV 4 = ? N 5 = ? JV 6 = [8] iV 7 = [1] =^> N 4 = [2] iV 5 = [5] iV 6 = [8] 7V 7 = [1] =^> 



7V 8 = [2 5] 



iV 9 = [1 ; 



iVio = [1258] 



JVi = ? 



iV 2 = [2 5] iV 3 = [8 1] 





iV 4 = [2] JV 5 = [5] iV 6 = [8] N 7 = [1] 



N 8 = [2 5] 



Ng = [1 , 



JVio = [1258] 



(9), 



N s = [2 5] 



iVg = [1 8] 



iVio = [1258] 
Ni = [2 5 8 1] 




iV 4 = [2] iV 5 = [5] iV 6 = [8] iV 7 = [1] 



N 8 = [2 5] 



N 9 = [1 8] 



A^io = [1258] 



Figure 1.8: Case 1 (out of 2) following the derivation of Figure 1.7 



every time a list L is split into two halves, the left half is always either of the same length as the right 
half (if L has even length) or else it is shorter by one (if L has odd length); it cannot possibly be longer. 
Hence, the N 6 list must be the one-element list [8]. 

• Step (3) infers that the Ng list must be [1 8]. This follows because the JVg list represents the result of 
merging A^g and N?, whose precise values are both known at this point. 

• Step (4) infers that the 7Y 3 list must be [8 1]. This follows because we already know the left and right 
halves of N$ to be [8] and [1], respectively. 

• Step (5) infers that the Ng list must contain 2 and 5. This holds by virtue of the principle mentioned 
above in connection with step (1): when L and L' converge in L", any number occurs in L" iff it occurs 
either in L or in L'. Therefore, since we know that 2 and 5 occur in ./Vio but not in A*g, they must occur 

iniV 8 . 
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JVi = ? 



N 2 = ? 



N s = [8 1] 



iV 4 = ? N 5 = ? 7V 6 = [8] 7V 7 = [1] =^ iV 4 = [5] iV 5 = [2] iV 6 = [8] 7V 7 = [1] ^ 



iVio = [1258] 



JVi = ? 



iV 2 = [5 2] iV 3 = [8 1] 





iV 4 = [5] JV 5 = [2] iV 6 = [8] N 7 = [1] 



(13) 




= [1258 






N g = [1 8] 



iVio = [1258] 
Ni = [5 2 8 1] 




iV 4 = [5] iV 5 = [2] iV 6 = [8] iV 7 = [1] 





Ng = [1 8] 



iVio =12 5 8 



Figure 1.9: Case 2 (out of 2) following the derivation of Figure 1.7 



• Step (6) infers that the Ng list must be precisely [2 5]. PI We already know that it must have at least these 
two elements. If it had more than two elements, then N\q would have to have at least five elements, 
given that (a) jV 10 is the result of merging N s and Ng, and that (b) N 9 has two elements. But N 10 has 
four elements, therefore 2 and 5 must be the only two elements of iV 8 , leaving [2 5] and [5 2] as the only 
two possibilites. But the second possibility cannot hold, since TVg must be sorted (recall that only sorted 
lists get merged). Hence, the Ns list must be [2 5], 

At this point we do not have sufficient information to determine unique values for the N4 and N$ lists. 
However, we can narrow things down to two possibilities: either 7V4 and N$ are [2] and [5], respectively; or 
else they are [5] and [2]. These are the only two alternatives that are concistent with Ns = [2 5], given that Ns 
represents the result of merging N4 and JV5. The reasoning in each case is as follows: 



12 The result of this step does not appear in Figure |l.7| for space reasons, but it is shown as the common starting point of the subsequent 
case analysis in Figure|1.8|and Figure|1.9| 
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Case 1 : In that case (Figure 1.8 1, we proceed to infer that the value of Nq, must be [2 5], since N4 and N$ are 
the left and right halves of N 2 . And then, since we know both N% and N3 we can determine the value of 
the input TVi to be [2 5 8 1]. 



Case 2 : In that case (Figure 1.9 1, we deduce that the value of N 2 must be [5 2], for the same reason we cited 



in the preceding case. Similarly, we can then conclude that the input list must be [5 2 8 1]. 
We are now entitled to infer that the original input list must be either [2 5 8 1] or [5 2 8 1]. 

1.7.2 Formalizing the puzzle as an instance of Vivid 

There are three steps to obtaining a particular instance of Vivid: 

1 . Specify an attribute structure A. 

2. Specify a vocabulary E, 

3. Specify an interpretation of the relation symbols of E into A as discussed in Section [L4| 
In the following three sections we carry out these steps in detail for the Mergesort puzzle. 

Specifying the attribute structure 

Let Node be the universe of nodes and let Z* be the set of all finite sequences (lists) of integers. An appropriate 
attribute structure for the Mergesort puzzle is the following: 

Am = (id : Node, children : Node*, data : Z* ; {R\, R 2 , R3, Ra, R5, R§, -R7} U {Rl \ L € Z*}) 
where the relations R\, . . . , R-j, Rl are as follows: 

• Ri C Node* x Node x Node, with 

Ri([ni ... n k ],n, n') <^> {n, n'} C {m, . . . , n k }. 

• R 2 C Node x Node* x Node* , with 

R 2 (n, [m ... n k ], [n'x . . . n' m ]) One {«i,. .. ,n k }(~) {n[, . . . , n' m }. 

• R3 C Z* x Z* x Z*, with 

i2 3 ([xi ... x k ],[yi ... y n ], [zi ... z m }) <&[x! ... x k ] = [y 1 ... y n z x ... z m ), 
i.e., iff [xi ... Xk] is the concatenation of [yi ... y n ] and [z\ ... z m ], 

• R 4 C Z* x Z*,with 

Ri([xi ... x k ],[yi ... y n \) <^n e {k,k+ 1}. 

• R5QZ*, with 

-R 5 ([xi . . . Xk]) »i. < x i+ i fori = 1, . . . ,k - 1, 

i.e., iff [xi ... Xk] is sorted. 
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• Re C Z* x Z* x Z*, with 

i? 6 ([zi ... Xk],[yi ■■■ y n ),[ z i ■■■ z m ])^{x 1 ,...,x k \ ^{yi,...,y n }yj{z 1 ,...,z m \. 

• R 7 C Z* x Z* x Z*,with 

R 7 ([xi ... x k ],[yi ... y n ],[zi ■■■ z m ])-^k = n + m. 

• R L <Z Z*,with 



R\ 



i(b/i 



2/nD^^i ■•■ a;&] = [yi 



[In] 



Note that we have infinitely many unary relations Rl, parameterized by L. Each such relation takes an arbitrary 
list of integers L' and tests for the equality L' = L. 

To make things concrete, Figure fTTTTIpresents an implementation of this attribute structure in SML. 

Specifying the vocabulary 

We have seven relations symbols: peak, valley, append, union, and sum are ternary; halves is binary; 
and sorted is unary. In addition, for each list of integers L we have a unary relation symbol valz,. We use 
Ni,N 2 , ■ ■ ■ as constant symbols and vi,v 2 , ■ ■ ■ as variables. 



Specifying the interpretation 



The interpretation of the relation symbols is shown in Figure [TTTO] 
More intuitive explanations follow: 

• peak(«i, ^2,^3) holds iff nodes v 2 and w 3 are both children of v\. 

vi 



v 2 v 3 

• valley(vi, v%, V3) holds iff v 2 and V3 are both parents of v\. 



Symbol 


Arity 


Realization 


Profile 


peak 


3 


Ri 


[(children, 1), (id, 2), (id, 3)] 


valley 


3 


R 2 


[(id, 1), (children, 2), (children, 3)] 


append 


3 


R 3 


[(data, 1). (data, 2), (data, 3)] 


halves 


2 


-R4 


[(data, 1), (data, 2)] 


sorted 


1 


R 5 


[(data, 1)] 


union 


3 


R§ 


[(data, 1), (data, 2), (data, 3)] 


sum 


3 


R 7 


[(data, 1), (data, 2), (data, 3)] 


vali 


1 


Rl 


[(data, 1)] 



Figure 1.10: The interpretation of the Mergesort puzzle vocabulary. 
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• append(ui,i>2,t>3) holds iff the list attached to node vi (i.e., the data field of Vi) is identical to the 
concatenation of the lists attached to nodes V2 and v$, respectively. 

• halve s(i>i, V2) holds iff the lengths of the lists attached to nodes v\ and 112 are approximately equal; 
more precisely, iff the length of the v 2 list is either equal to or one more than the length of the V\ list. 

• sorted(«i) holds iff the list attached to node V\ is sorted. 

• union(i>i, V2, V3) holds iff the list attached to node V\ contains all and only those elements that occur 
either in V2 or in U3 (or in both). 

• sum(i>i, V2, W3) holds iff the length of the V\ list is equal to the sum of the lengths of the v 2 and v 3 lists. 

• vali(wi) holds iff the list attached to node V\ is identical to L. We write val(vi, L) as an abbreviation 
for vali(«i). 



datatype Nat = zero | succ of Nat; 

datatype Node = node of Nat; 

fun member (x,L) = List. exists (fn y => x = y) L; 

fun subset (LI , L2 ) = List. all (fn x => member (x, L2 ) ) LI; 

fun Rl(L,nl,n2) = member (nl,L) andalso member (n2 , L) ; 

fun R2(n,Ll,L2) = member (n, LI) andalso member (n, L2 ) ; 

fun R3(L1,L2,L3) = LI = L20L3; 

fun R4(L1,L2) = let val lenl = length LI 
val len2 = length L2 
in 

len2 = lenl orelse len2 - lenl 1 1 
end; 

fun R5 ( [ ] ) = true 

I R5(x::L) = R5 (L) andalso List. all ( f n y => x <= y) L; 

fun R6(L1,L2,L3) = let val L = L20L3 
in 

subset (LI, L) andalso subset (L, LI) 
end; 

fun R7(L1,L2,L3) = length (LI) = length (L2) + length (L3); 



Figure 1.11: SML code implementing the attribute structure of the MergeSort puzzle. 
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1.7.3 The formal proof 

The following Horn clauses are all the axioms we need for solving Mergesort puzzles. Their meaning should 
be clear in light of the foregoing interpretation. 

Vi>i,U2j^3 • peak(vi, ^2, V3) =Mialves(u2, W3) halves-axiom 

V vi, V2,vz . valley(ui, «2, W3) =>■ sorted(vi) A sorted^) A sorted^) sorted-axiom 

V i>i, v%, V3 . valley(«i, V2, V3) V peak(vi, V2, V3) =>• union(i>i, t>2, V3) union-axiom 

Vwi,W2,t | 3 ■ peak(ui,i;2, V3) =► append(i>i, V2, V3) append-axiom 

Vui,V2)W3 . valley(wi, W2, W3) => sum(«i,ti2, U3) sum-axiom 

Now let nodei, . . . ,nodeio be ten nodes from the universe of all nodes, Node. In combination with the 
attribute structure Am, these ten nodes constitute a system. The diagrams shown in Figure [L7j Figure 1.8 and 



Figure [L9| depict specific named states of this system. Consider, for instance, the starting diagram, at the upper 
left corner of Figure 1 1.7 1 This represents a named state (a; p), where the partial constant assignment p is 

N± h+ nodei, N2 1— * node2, ■ ■ ■ , -/V10 i— ► nodeiQ (1.109) 

(with p(Ni) undefined for i > 10); while the two ascriptions children and data are as follows (the id ascription 
is defined in the obvious way): 

children(node\) = [node2 nodes] 
children(node^) = [nodes] 

children(node 10) = \\ 

and 
data{node x ) = {[}, [1], [2], [5], [8], [1 2], [1 5], . . . , [8 5 1], . . . , [1 2 5 8]} 

data(node 9 ) = {[8], [1 8], [8 1], [2 8], . . . , [5 8 1], [2 5 1 8], . . .} 

data(nodeio) = {[12 5 8]}. 

Observe the equation for data(node\). At this point we do not know anything about what list appears at 
node\ (a complete lack of knowledge signified by the inscription N\ = ?), so the data field of node\ is entirely 
unconstrained: it contains all possible lists of length four obtained by permutations of four objects taken four at 
a time (P(4, 4) = 4! =24 total); plus all possible lists of length three obtained by permutations of four objects 
taken three at a time (P(4, 3) = 24 total); plus all possible lists of length two obtained by permutations of four 
objects taken two at a time (P(4, 2) = 12), plus all possible lists of length one (4), plus the empty list, for a 
sum total of 24 + 24 + 12 + 4 + 1 = 65 different lists. The data ascription maps every "questionmark node" 
(e.g., the nodes labeled by Nq or Ng) to the same set of 65 lists. Hereafter we will denote this set of 65 lists by 
C. By contrast, the data ascription for node 9 (the node labeled by Ng) is subject to the constraint that all list 
values must contain 8, so this narrows down the possibilities to a total of 24+ 18 + 6+ 1 = 49. Further down, 
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T2 by thinning with union-axiom; 

ts by thinning with halves-axiom; 

T4 by thinning with union-axiom, sorted-axiom; 

tb by thinning with append-axiom; 

t§ by thinning with union-axiom; 

t-j by thinning with sum-axiom, sorted-axiom; 

cases from union-axiom, halves-axiom'. 

t& —> T9 by thinning with append-axiom; 
rio by thinning with append-axiom; 
observe val(JVi, [2 5 8 1]) V val(JVi, [5 2 8 1]) 
ri2 — > Ti3 by thinning with append-axiom; 
Ti4 by thinning with append-axiom; 
observe val(iVi, [2 5 8 1]) V val(iVi, [5 2 8 1]) 



Figure 1.12: Formal Vivid proof solving the Mergesort puzzle of Section 1.7.1 



the value of afata for nodeio is completely determined — the singleton { [1 2 5 8]}.Fj The named system state 
corresponding to any of the diagrams shown in connection with the Mergesort puzzle is likewise defined. The 
children ascription and the constant assignment remain the same in every case; while the data value is specified 
in accordance with the preceding conventions. 

Extracting the appropriate system state from a given diagram can be viewed as the task of computing a 
parsing function <p that takes a concrete two-dimensional representation and produces an abstract syntax tree 
for it. Conversely, reconstructing a diagram from the underlying system state can be seen as computing an 
"unparsing" function ip that proceeds in the reverse direction, rendering system states graphically. As with 
customary parsing and unparsing, we have 



ip((j>(d)) = d and </>00)) = a 



(1.110) 



for all diagrams d and system states a, where the first identity is understood to obtain up to topological equiva- 
lence. Fj From a practical standpoint, most of the effort required to build a Vivid language would be alloted to 
the implementation of these two functions. In the case of the Mergesort puzzle, both <f> and ip can be computed 
efficiently — in low polynomial time — using standard graph-theoretic algorithms. 



Finally, Figure 1.12 shows the formal Vivid proof that solves the Mergesort puzzle discussed in Sec- 



tion 



1.7.1 We conclude with a detailed analysis of this proof. 



First, we need a simple lemma: 

V vi, V2,vz . valley(ui, ^2,^3) =>■ union(vi, V2, V3) A sum(wi, V2, V3) [lemma] 



13 These are unnecessarily coarse approximations. We could leverage our knowledge of the domain to further cut down the possibililities 
drastically. For instance, we know that at the top node only lists of length four could appear — or, in general, only lists of the exact same 
length as the unique list that appears at the bottom node representing the output. Further, we know that if any node has only lists of n items 
as possible values, then the left and right children can respectively only have lists of length [ n /2j and [n/2] as possible values, and so 
on. In this manner cardinality constraints would propagate down the graph and significantly curtail the values of the data ascription. This 
would be important for an efficient implementation of the Mergesort puzzle, but it is not necessary for our present purposes. 

14 Diagrammatic identity in general can be a vague notion (e.g., when exactly can we say that two drawings depict the same mountain 
range?) and this is part of the reason why logicians and mathematicians have had a skeptical attitude towards diagrams (Quine's dictum 
"No entity without identity" (Quine 1969) comes to mind). Nevertheless, there are many cases where we can formulate rigorous necessary 
and sufficient conditions for two diagrams to be considered identical, using topological or other extensional notions. 
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This can be derived from our five axioms in a few lines of Vivid, by some elementary sentential reasoning; we 
leave the derivation to the reader. 

Next, let <j\ , . . . , C6 be the system states corresponding to the six diagrams that appear in Figure[L9]starting 
from the top left corner and proceeding clockwise, so that c, represents the graph to the left of the arrow 
indicating the i step. Likewise, let 177, . . . , a 10 and an, . . . , CT14 be the states corresponding to the diagrams 



of Figure 1.8 and Figure 1.9 respectively. For any i — 1, . . . , 14, we write n to denote the named state (a*; p), 



where p is the constant assignment ( 1.109 1 



Recalling that composition is right-associative, we see that the proof in Figure [L12| is a sentential proof D, 
as it is of the form 

D = A i; --- ;A 6 ;Zy, 

i.e., a composition of six diagrammatic steps Ai, . . . , A$ followed by a sentential deduction D' of the form 

cases from ^ l7 ...,F fc : (01; p x ) -» D x \ ■ ■ ■ \ (a n ;p n ) -* D n , 
a diagrammatic-to-sentential case analysis. The starting point for the proof is the context 

7i = (/3i;n), (l.iii) 

where j3 1 contains the five universally quantified clauses of our axiomatization and the aforementioned lemma. 
This is the context in which the entire proof D will be evaluated. 
Let us why the first step Ai, the diagrammatic inference 

T2 by thinning with union-axiom 



succeeds. According to the semantics of thinning (Figure 1.3 1, this step will be valid provided that 



Tl "' {union-axiom} T2 ' 

i.e., provided that n entails r-i with respect to union-axiom. This means that every alternative way of extending 
T\ w.r.t. T2 must falsify union-axiom (for an arbitrary variable assignment). More precisely, it must be the case 
that for every named state r = (<j; p) such that Alt{ri, T2, t) we have 

iV a . p \ 1 y {union-axiom) — false (1.112) 

for all x- Pick any such r. Since T\ and t-i share the same constant assignment p, the only way r can be an 



alternative extension of n w.r.t. r 2 is if we have Alt(a%, &%, a) (Definition 10 1. The only state a that qualifies 
as such an alternative is the one that is identical to G\ except that the data ascription maps node§ to the set of 



all lists in C that do not contain 8. It is easy to see that (|1.112 1 holds in that state. Indeed, consider an arbitrary 



X- By ( 1.5 I, union-axiom will be false in (er; p) and x if there are some nodes node il , node^, and node i3 such 
that 

I( ffi p)/ x[vi ^node n ,v 2 ~node t2 ,v 3 ^node z;t ]^^y{v^v 2 ,v z ) V peak(«i, v 2 , v 3 ) => union(«i, v 2 , v 3 )) = false. 

Let these three nodes be nodeg, nodes, an d nodej, respectively (i.e., the nodes labeled by Ng, Nq and 7V7). 
For these nodes we clearly have: 

I{<T;p)/x[vi^node 9 ,v 2 ^node 6 ,v 3 ^node 7 ](valley(vi,V2 ) V3) V peak(ui, V 2 ,v 3 )) = true 

(since the nodes form a valley) and yet 

I (a;p)/xlv 1 ^node 9 ,v 2 ^node 6 ,v 3 ^node 7 ]( uniori ( v l,V2,V 3 )) = false. (1.113) 
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(1.113 I holds because for every list L in the data field of nodeg in a and for every list L' in the data field of 



nodes in a and every list L" in the afafa field of node*; in a, we have 

-^R 6 (L,L',L"), 

the reason being that every such L contains 8 but no such L" contains 8 (because data(node 7 ) in a contains 
only one list value, [1]) and no such L' contains 8 (by virtue of a being an alternative extension of a\ w.r.t. 
o- 2 )- 

It is important to note that in practice these three nodes would be discovered automatically by exhaustive 
search. Specifically, the system would evaluate the formula union-axiom in the named state (a; p)and an 
arbitrary variable assignment x^jto determine if it comes out false. Now a universally quantified formula 
such as union-axiom is evaluated in a given \ by binding the universally quantified variable to successive 
system objects and recursively evaluating the body in the updated \- If the body comes out false for some 
system object, the whole formula is deemed false. If the body itself is another universally quantified formula 
then we have more choice points and possible backtracking. In the worst case for the puzzle example, the 
evaluation of union-axiom will need to examine 10 3 = 1000 difference possible assignments of variables to 
objects, since the system comprises 10 nodes and the formula has three universally quantified variables. In 
such a worst-case scenario, the body of union-axiom would be evaluated for each of the 1000 node triples. 
For most of these triples, union-axiom would come out unknown because there is not enough information to 
enable a definitive judgment. Consider, for instance, the evaluation of the body of union-axiom in the triple 

x[vi | — > node\, i>2 <— > node2, t>3 <— > node^]. 

While it is true that nodei,node2, and node?, form a peak, we have 

I (a;p)/xlv 1 ^node 1 ,v 2 ^node 2 ,v^node 3 ]( union (vi,V2,V 3 )) = unknown 

because, in <j, the realization of union, Rq, holds for some list values in the corresponding data fields of 
nodei,node2 and node% and does not hold for others (see ( |1.3) ). 

In this particular example we have 10 system objects and the most populous attribute value has 65 elements, 
so a formula such as union(wi, V2, v%) could, in theory, take up to 65 3 = 274, 625 evaluations to settle. Com- 
bined with the 1,000 triple possibilities dictated by three universal quantifiers, we could look at the non-trivial 
number of 274, 625, 000 evaluations. However, in practice atomic formulas such as union(-y 1; w 2 , v 3 ) would 
be settled speedily because for most node assignments we would get some true and some false values, quickly 
leading to an unknown result. So even the worst case of 1000 different evaluations is not computationally 
formidable. 

Nevertheless, we observe that the user can always improve the efficiency of the proof checking by providing 
more information in the proof — information that guides the searh in the right direction. For example, we could 
replace the first step 

T2 by thinning with union-axiom 

by the following sequence of steps: 

specialize union-axiom with Ng,Ng, N 7 \ 

observe valley(A^ 9 ,7V 6 , N 7 ); 

right-either peak(iV 9 , N e , N 7 ) V valley(7V 9 , N 6 , N 7 ); 

modus-ponenspeak(7V 9 ,iV 6 ,7V 7 ) V valley(7V 9 , jV 6 ,iV 7 ) ^> U nion(N g ,N 6 ,N 7 ), 

peak(JV 9; N 6 , N 7 ) V valley(7V 9 , N 6 , N 7 ); 
r 2 by thinning with union(N g , N e , N 7 ) 



15 This is legitimate by virtue of Lemma 2 
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Here we focus directly on the three nodes of interest by citing un±on(Ng, Nq, N7) as the justification of 
the thinning step, instead of citing the universally quantified union-axiom. By eliminating the three universal 
quantifiers, we avert the need to evaluate the body of union-axiom over all possible triples of nodes. The 
tradeoff is a typical manifestation of the usual tension between brevity and efficiency: a very brief proof takes 
large steps whose verification can be difficult because it requires search; whereas a detailed proof takes small 
steps that are easy to check because they involve little or no search. We can always buy efficiency at the expense 
of consiceness. 

Returning to the proof, let us examine the second step: 

T3 by thinning with halves-axiom. 

As with the previous application of thinning, this step is valid only if 

T2 {halves-axiom} T 3' 

meaning that any named state r = (a; p) that is an alternative extension of t 2 w.r.t. 73 must falsify halves-axiom. 
As before, because the constant assignment does not change, the only way we can have Alt(r 2 , T3,t) is if we 
have Alt(cr 2 , CT3, cr). And given that in a 2 the data field of node e contains all and only those lists that contain 
8, a is an alternative extension of a 2 w.r.t. 0-3 iff it is a list in C that contains 8 and has length greater than one, 
e.g., [2 5 8]. But in that state halves-axiom is falsified (with node^, node$, and node-? providing the counterex- 
ample peak), hence the thinning step is sanctioned. Similar rationales justify the next four thinning steps. We 
encourage the reader to work through them rigorously. 

We come finally to the case analysis, which turns on the claim that from the state 07 and on the basis of the 
lemma, there are only two possible states, cr 8 an d cr i2- Symbolically, 

(<7 7 ;p) \h {lemma} {(c7 8 ;p),(ai2;p)}. (1.1 14) 



Consulting Definition 1 1 we see that ( 1 . 1 14 1 holds iff for every (a'; p 1 ) such that 



Alt((a 7 ; p), {(<r s ;p), (crias p)}, K //)) (1.115) 

we have If ff i. j\i^(lemmd) = false for all \- Again, because the constant assignment does not change, dl.ll5l 

holds iff ' 

Alt(a 7 ,{a s ,a 12 },a') (1.116) 



(by Definition 10 1 



Now there are two alternative extensions of a 7 w.r.t. {<X8,<7i2}: one, call it a a, in which we keep the 
{[2], [5]} value of node 4 steady but complement it for node 5; while the other, call it as, is one in which we 
complement the data value of node 4 in 07 and retain the data value of node 5. The relevant parts of both states 
can be depicted graphically as follows: 

N 2 =? N 9 = ? 

c^a- 1 cr B : 



JV 4 = {[2],[5]} N 5 =C\{[2),[5}} 2V 4 = £\{[2],[5]} ^5 = {[2], [5]} 



7V 8 = [2 5] N 8 = [2 5] 

A routine calculation will confirm that both possibilities falsify the cited lemma. 
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1.8 Related Work 

We have derived much inspiration from the seminal work of Barwise, Etchemendy, and others on Hyperproof 
(Barwise and Etchemendy 1995b). One of the chief contributions of Hyperproof was its emphasis on incom- 
plete information and its ability to reason about ambiguous (partially determined) situations. These choices are 
not only pedagogically sound, since there are many types of reasoning problem^] in which students are given 
an incomplete sketch and are asked to fill in the gaps by way of inference; but they are also apt design choices 
for visual reasoning systems in general, since oftentimes the information that agents extract from a perceived 
image is incomplete, either because parts of the image are visually unclear or because they are not sure how to 
interpret themp] 

Important differences between Vivid and Hyperproof include the following: 

1. Hyperproof is specifically built for reasoning about simple blocks worlds. Vivid, by contrast, is a 
domain-independent framework. 

2. Hyperproof's treatment of incomplete information is limited and ad hoc. For instance, although a dia- 
gram can signify that the size of a block is unknown, it has no way of indicating that it is, say, large or 
medium but not small. By contrast, Vivid's mechanism for handling incomplete diagrammatic informa- 
tion via arbitrary sets of values is completely general. 

3. Vivid is based on the key DPL ideas of representing assumption scope with context-free block structure 
and formalizing the denotation of a proof as a function over assumption bases. These two ideas have 
several advantages for formalizing Fitch-style natural deduction (Arkoudas 2000, Arkoudas n.d.a). The 
standard Fitch practice — adopted by Hyperproof — of capturing assumption scope by drawing nested 
vertical lines might be viable for pedagogical purposes but would not scale to realistic proofs any more 
than using vertical lines to represent lexical scope in programming languages (instead of the usual begin- 
end pairs or curly braces) would scale to realistic programs. 

4. Vivid has a formal big-step evaluation semantics in the style of Kahn and Plotkin (Kahn 1987, Plotkin 
1981). This is not to say that Hyperproof does not have precise semantics or that its semantics cannot 
be formally defined; only that it does not draw on the same techniques from the field of programming 
language theory. We stress that this is not an issue of mere stylistic differences in presentation. Casting a 
formal semantics in a style such as we have used carries significant advantages, especially in metatheo- 
retic investigations, where many arguments take the form of neat induction proofs on derivations (witness 
our soundness proof). In general, such a semantics is an invaluable tool for reasoning about proofs in 
the system, and for evaluating the correctness of algorithms that manipulate such proofs p| 

5. Because it is based on DPLs, Vivid could be extended from its present form as a proof-checking frame- 
work into a Turing-complete programmable system allowing the user to formulate arbitrary tactics {meth- 
ods) combining diagrammatic and sentential inference steps, in such a way that the soundness of the 
methods would be guaranteed by the formal semantics of the language (see (Arkoudas n.d.c) for an ex- 
ample of how such extensions are actually performed). It is not at all clear how Hyperproof could be 
made programmable, let alone in a way that would guarantee soundness. 



16 E.g., in logical and analytical reasoning problems of standardized tests such as GRE or LSAT. 
17 As Konolige and Meyers (Myers and Konolige 1995) put it: 

When generating maps from perceptual input, noise or faulty sensors may both cause objects of interest to go undetected 

and leave analogical relations only partially determined. 

18 For instance, very efficient proof-simplification algorithms that were developed for J\fT>C (Arkoudas n.d.b) were made possible — and 
proven sound — owing to the formal operational semantics of the language. The same ideas could be incorporated into VIVID, resulting in 
general principles and procedures for eliminating redundant reasoning from diagrammatic proofs. 
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6. Hyperproof is proprietary; Vivid is in the public domain. The difference is not without practical ramifi- 
cations. The open design of Vivid enables highly modular implementations because it exposes a sharp 
separation between the purely graphical tasks of diagram parsing and unparsing on one hand and the 
system's syntax, semantics, and underlying diagrammatic inference procedures on the other. The latter 
are fixed once and for all and proven sound. All one needs to do in order to implement a specific instance 
of Vivid is fix a class of diagrams and provide a diagram parser (compiling diagrams into system states) 
and unparser (rendering system states graphically). Hyperproof is much more of a monolithic black box, 
and any attempt by third parties to build Hyperproof-like systems for other domains would have to resort 
to reverse engineering. 

The work of Konolige and Myers on "reasoning with analogical representations" (Myers and Konolige 
1995) is somewhat similar in spirit to our research, in that it seeks to formulate domain-independent principles 
of diagrammatic reasoning. However, they do not provide any linguistic abstractions for performing such 
reasoning. Rather, they outline a set of data structure operations (which they call "the integration calculus") that 
can be used to integrate diagrammatic inference into existing reasoning systems, and which can be described 
as a programming interface. By contrast, we have introduced a specific, precisely defined family of languages 
for heterogeneous natural deduction, with novel syntax forms and formal semantics. Further, our method 
for dealing with what they call "structural uncertainty" (incomplete diagrammatic information) is much more 
general. Finally, our system is strictly more powerful in that it can perform diagrammatic case reasoning; their 
integration calculus does not have that capability. 

DIAMOND (Jamnik 2001) is a system for checking diagrammating proofs of certain types of arithmetic 
theorems. The system is designed to reason exclusively aboyt natural numbers, and specifically with universally 
quantified identities of the form V • • • . S = t, where s and t are terms built from the numerals 0,1,2,..., 
variables, and operators such as addition, multiplication, etc. A typical example is the identity asserting that 
the sum of the first n odd natural numbers is n 2 , symbolically written as 



E 2 * 



n z (1.117) 



Diagrammatic proofs are only given for particular instances of the theorem, e.g., for ( 1 . 1 17 1 one might give a 
diagrammatic proof for n = 4, establishing that 1 + 3 + 5 + 7 = 4 2 = 16. A diagrammatic proof of such a 
concrete identity is given by representing both terms (1 + 3 + 5 + 7 and 4 2 ) as diagrams, and then rewriting 
both diagrams to a common form. This clearly depends on the system's ability to represent concrete numeric 
terms by suitable diagrams. This is possible and indeed intuitive for certain types of terms. E.g., 4 2 can be 
represented as a 4 x 4 square matrix of dots: 



and likewise for any n 2 . It is not so easy for other terms, however, and indeed DIAMOND currently cannot even 
express some arithmetic theorems. 

After the user has successfully carried out several diagrammatic proofs of such concrete instances of the 
identity in question, the system uses inductive learning techniques in an attempt to automatically extrapolate a 
schematic proof algorithm capable of taking any number n and proving the identity for that particular number. 
If successful, the schematic proof algorithm then needs to be proved correct in a metatheoretic framework. This 
is probably the most problematic step of the process, as the problem is undecidable in general. We are thus 
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faced with the somewhat odd consequence that even though DIAMOND is only a proof checker and not a proof 
finder, it might nevertheless still fail to yield a verdict. Therefore, it might make more sense to incorporate 
abstraction devices into the diagrams in a disciplined way, and attempt from the outset to give diagrammatic 
proofs of the general form of the theorem, instead of insisting on dealing with concrete diagrams only. 

GROVER (Barker-Plummer and Bailin 1992) is a theorem-proving system that uses diagrams to guide 
the proof search. The system consists of a conventional (sentential) automated theorem prover (ATP), &, 
augmented with a diagram processor. The diagram processor examines the given diagrams and, based on 
the extracted information, it constructs an appropriate proof strategy for &. Its authors report having used the 
system to obtain automatic proofs for the diamond lemma, as well as for the Schroder-Bernstein theorem of ZF. 
Both are non-trivial results; the Schroder-Bernstein theorem, in particular, has a quite sophisticated sentential 
proof that is far from even the current state-of-the-art in ATP technology. In their view, a diagram represents a 
trail of the objects that are involved in the proof, along with key properties of and relation among such objects. 
This is an interesting view of diagrams, but it differs from the (more rigorous) sense in which diagrams are used 
in systems such as Hyperproof or Vivid, where diagrams are essentially used as visual premises and inference 
rules are applied to them in the usual step-by-step fashion. 

Anderson and McCartney (Anderson and McCartney 2003) present IDR, a system for representing and 
computing with arbitrary diagrams. A diagram is viewed as a tesselation of a finite two-dimensional planar 
area, with each tile having a unique triple of numbers i, j, k associated with it, indicating a value in the CMY 
(Cyan, Magenta, Yellow) color scale. Apart from the spatial relationships between the tiles, the meaning of 
a diagram is captured mainly via tile coloring, with different colors (or shades of gray) representing different 
types of information. They introduce a set of operations on diagrams, each of which takes a number of input 
diagrams of the same dimension and tessellation and produces a new diagram in which the color value of a tile is 
some function of the color values of the corresponding tiles of the input diagrams. Among other applications, 
IDR has been used to solve the n-queens problem diagrammatically, to induce correct fingerings for guitar 
chords, and to answer queries concerning cartograms of the USA. The system is more concerned with diagram 
computation rather than with inference; there are no general notions of entailment, soundness, etc. IDR is also 
not heterogeneous. It is exclusively diagrammatic, in that all the available operations are applied to diagrams, 
not to combinations of diagrams and symbolic information. 

1.9 Conclusions 

A cursory reading of this paper might leave one asking: "So where are the diagrams? All I see are sets and 
lists and functions and so on — the usual sentential stuff." Indeed, as Greaves (Greaves 2002) correctly states: 

Diagrammatic representations can be recognized by the extent to which the geometric properties 
of the components of the representation are relevant to their interpretation, and the ways in which 
these properties impact the reasoning methods which are licenced by the overall theory. 

But our theory revolves around attribute structures, system states, etc., and has ostensibly little to do with 
"geometric properties" of any kind. 

There is nothing odd about that. Our theory is a logical analysis of the computational and information- 
theoretic aspects of certain types of diagrammatic reasoning. It is not itself a piece of diagrammatic reasoning, 
nor does it need to be. A mathematical analysis of visual inference does not itself need to be visual any more 
than a mathematical analysis of acoustics needs to be musical, or any more than a mathematical analysis of 
heat needs to be hot. 

Still, one might wonder whether any representation of diagrams by set-theoretic structures is not bound 
to lose something of the essentially pictorial nature of diagrams. Perhaps, but the issue is rather orthogonal 
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to our concerns. Our analysis is mostly motivated by engineering concerns and is therefore given with a 
view to building robust, efficient, usable systems that permit perspicuous heterogeneous proofs combining 
diagrammatic and sentential reasoning. 

In summary, we have introduced Vivid, a family of denotational proof languages (DPLs) that combine 
sentential and diagrammatic reasoning in a Fitch-style natural deduction framework. Vivid is based on the no- 
tion of attribute systems, and on the use of Kleene's strong three-valued logic to interpret first-order signatures 
into attribute structures. To obtain a particular instance of Vivid, we need only specify an attribute structure, a 
signature, and an interpretation of the signature into the structure. 

We have not discussed how diagrams would be concretely represented within the proof text. That is an 
interface issue, not an issue of abstract syntax or semantics. One possibility would be to give names to diagrams 
and then have those names appear in the proof text, but with hyperlinks. If a user clicks on such a link, a picture 
depicting the corresponding diagram would pop up, and the user could view or edit the diagram as necessary, 
save it as another diagram, etc. Of course, as we have already stressed, how diagrams are drawn depends on 
the specific application domain at hand; it is completely separate from all other aspects of the language. This 
modularity could be put to good use, e.g., an implementation of Vivid could be designed as an SML functor 
(Paulson 1996) that will take an attribute strucure A; the interpretation of a signature £ into A; and a drawing 
module that can draw an arbitrary ,4-system; and will output a parser and an interpreter, i.e., a proof checker 
for the instantiated language. 

Introducing names brings up another possibility. As it stands, an implementation of Vivid would be a 
type-a DPL, i.e., a proof checker: it would accept a proof combining sentential and diagrammatic steps and 
would either pronounce it sound or else point out a reasoning error. If we introduce unrestricted naming and 
computation, we can make these into type-u DPLs (Arkoudas n.d.c, Arvizo n.d.), capable not only of proof 
checking but of arbitrary proof search as well. It would be very interesting to see what types of methods can 
be written in such a setting for the purpose of automating diagrammatic inference, and exactly what type of 
formal soundness guarantee we might be able to provide. 

Another important issue is efficiency. Depending on the system we are working with, we may need ex- 
ponential time in the size of the attributes to check whether an application of a rule such as thinning is valid. 
This is due solely to the size of the attributes and is orthogonal to how "large" are the steps taken by the user. 
Even if the user takes a very small step, say to exclude one possible value from a set thereof, we may still 
need to explore exponentially many subsets. Two possibilities for ameliorating this issue are: (a) representing 
sets of attribute values by binary decision diagrams (BDDs) (Bryant 1992), and (b) symbolic evaluation. For 
(a), it is hoped that a compact representation of the relevant subsets might speed up rule checking. There are 
standard techniques for representing an arbitrary subset 5" of any finite set S by a BDD, basically by encoding 
the characteristic function of S' as a Boolean function (Huth and Ryan 2000). With symbolic evaluation, we 
may be able to prune very large parts of the search tree if we incorporate a modest degree of domain knowl- 
edge into the search process. For instance, if we determine that a time (hi, mi) of a clock c\ is not ahead of 
some clock c 2 , there is no point in trying other possible times (h\, m^) of c\ if h[ < hi or if h\ = hi and 
m! x < mi. Sophisticated techniques for performing symbolic predicate evaluation (similar to the symbolic 
evaluation methods in model checking (Clarke et al. 1999)) could have a significant payoff. 
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